Position calculation system

ABSTRACT

Embodiments employ one or more ellipses to determine distance and location in relation to a camera. The captured images of ellipses are analyzed to determine the size of the major axes of the ellipses in the captured image. From the size of the major axis, using mathematics, the distance from the camera can be computed. This distance from the camera combined with the location of the center of the ellipse in the captured image may then be used to locate the ellipse in X-Y-Z space. The ellipses may be placed on articles attached to the body or head of a person, such as a hat or glasses.

This application is a continuation of U.S. Utility patent applicationSer. No. 16/865,623 filed 4 May 2020. U.S. Utility patent applicationSer. No. 16/865,623 is a continuation of U.S. Utility patent applicationSer. No. 15/992,062, filed 29 May 2018, issued as U.S. Pat. No.9,986,228 which is a continuation-in-part of U.S. Utility patentapplication Ser. No. 15/461,391, filed 16 Mar. 2017, issued as U.S. Pat.No. 9,986,228, which claims the benefit of U.S. Provisional PatentApplication 62/312,937, filed 24 Mar. 2016, and of U.S. ProvisionalPatent Application 62/354,787, filed 26 Jun. 2016, and of U.S.Provisional Patent Application 62/360,296, filed 8 Jul. 2016, and ofU.S. Provisional Patent Application 62/393,083, filed 11 Sep. 2016, andof U.S. Provisional Patent Application 62/414,676, filed 29 Oct. 2016,and of U.S. Provisional Patent Application 62/414,678, filed 29 Oct.2016, and of U.S. Provisional Patent Application 62/435,831, filed 18Dec. 2016, and of U.S. Provisional Patent Application 62/446,524, filed15 Jan. 2017, the specifications of which are hereby incorporated hereinby reference. U.S. Utility patent application Ser. No. 15/992,062 isalso a continuation-in-part of U.S. Utility patent application Ser. No.15/883,277, filed 30 Jan. 2018, which is a continuation of U.S. Utilitypatent application Ser. No. 14/547,555, filed 19 Nov. 2014, issued asU.S. Pat. No. 9,883,173, which claims the benefit of U.S. ProvisionalPatent Application 62/035,477, filed 10 Aug. 2014, and of U.S.Provisional Patent Application 61/934,806, filed 2 Feb. 2014, and ofU.S. Provisional Patent Application 61/920,755, filed 25 Dec. 2013, thespecifications of which are hereby incorporated herein by reference.

U.S. Utility patent application Ser. No. 15/992,062 is also acontinuation-in-part of U.S. Utility patent application Ser. No.14/106,766, filed 15 Dec. 2013, which claims the benefit of U.S.Provisional Patent Application 61/897,983, filed 31 Oct. 2013 and U.S.Provisional Patent Application 61/900,982, filed 6 Nov. 2013, thespecifications of which are hereby incorporated herein by reference.

BACKGROUND OF THE INVENTION Field of the Invention

One or more embodiments of the invention are related to the field ofoptical systems for producing views of a display, such as for examplestereoscopic views or views that vary based on a user's location. One ormore embodiments are also related to the field of tracking systems forheads or eyes. More particularly, but not by way of limitation, one ormore embodiments of the invention enable a quad view display system.

Description of the Related Art

There are various methods known in the art for creating a 3Dstereoscopic image. Most commonly these include, but are not limited toshutter glasses, passively polarized glasses, and anaglyph glasses. Thestereoscopic image may present on a flat panel, projection or otherdisplay medium. These glasses discriminate between first and secondimages and coordinate with the display medium to present thecorresponding image to the correct eye. In this way, a stereoscopicimage or a dual view image may be presented.

In addition to or instead of presenting different images to differenteyes of a single user, in many situations it is useful to presentdifferent images to different users. While shutter glasses may be usedfor this purpose, the speed of the shutter lenses and the speed ofswitching the display panel limits the number of concurrent users. Thereis a need for a system that supports a larger number of views of asingle shared display.

In many situations, it is also useful or necessary to track the positionand orientation of each user's head or eyes, for example to present animage to that user that corresponds to the user's viewpoint. Systemsthat are used to track the eyes in order to produce 3D stereoscopicimages vary, but there are no inexpensive methods currently in use.Examples of tracking systems known in the art include the ZSpace system,the TrackIR system by Naturalpoint, and the Microsoft Kinect system.Each of these devices has limitations.

ZSpace employs a tablet with two custom cameras that track glasses with5 tracking blobs on the glasses. A version of this method is describedin US 2013/0128011. The devices are expensive; at the time of thiswriting the minimum order is ten units at a price between $22,000 to$47,000. This is cost prohibitive for the average user.

The TrackIR made by Naturalpoint employs infrared (IR) lights mounted onheadgear. The least expensive version is currently sold for about $200.The IR lights are difficult to attach correctly and the system isdifficult to employ correctly.

The Microsoft Kinect employs a method of structured light where patternsof IR light are shined on an object and the changes in the pattern dueto the shape of the object are used to determine the 3D structure andlocation of the object. These systems may employ one or more sources oflight and one or more cameras for viewing. The Kinect has the ability toface track, but results are unreliable. The current software is notoptimized for when glasses are worn and it loses tracking easily. Inaddition, the cost is around $200 for a Kinect.

Most modern computers already come equipped with a camera at no extracost. Those that do not may be equipped with an inexpensive camera thattransfers data via the USB port. A user would prefer to use the camerathat came with their computer rather than purchase an expensiveaccessory in order to create real world 3D stereoscopy. However, theseinexpensive cameras do not have depth sensing.

A series of markers or blobs on the glasses may be used to marklocations on the glasses. The distance between the blobs as measured bycamera angle may not yield accurate information because the head may beturned from the camera in several axes. This changes the distancebetween the blobs as seen by the camera. For example, turning the headto either side by thirty degrees could result in a fifty percent errorin the distance between two blobs located horizontally from one another.This results in a corresponding error in tracking location.

US 2010/0103516 describes a head tracking system employing reflectiveblobs that fails to take into account the error associated with turningthe head along various axes. It describes changing the polarization ofthese blobs by means of a retarder. The fact that these reflective blobsmay have their polarization altered has no bearing on this error.

Various methods have been proposed to track eyewear, most involvingmarkers or blobs on a surface. The shape of these systems varies withthe angular relationship of the surface to the camera(s). This makes itdifficult to track said eyewear accurately with inexpensive cameras.

Therefore, it would be highly desirable to have a system and/or methodthat uses one or more inexpensive cameras to track the head or the eyes,and that also negates the error due to tilting or turning of the head.

Another current problem involves manipulation of the 3D stereoscopicimagery. Z-space currently employs a pointer that is tracked andprovides a 3D line that is drawn from the tip of the pointer by thesoftware and is employed to manipulate stereoscopic images. The pointerdoes not produce a beam, but rather the beam is presented as a 3Dstereoscopic image that gives the impression of being generated from thepointer. This requires a complex system of two precisely positionedcameras to precisely track the pointer. Thus, it cannot be used oncurrent common systems that have only one camera. In addition, one handis used to hold the tablet device and the other is used to manipulatethe pointing device. This leaves no hands free to manipulate othercontrols. It would be desirable to have a method of pointing and ofmanipulating imagery that requires only a single camera, and that doesnot require a separate pointing device.

For at least the limitations described above there is a need for a quadview display system.

BRIEF SUMMARY OF THE INVENTION

One or more embodiments described in the specification are related to aquad view display system. Embodiments of the invention may includeglasses that select one or more views generated from a display that maybe shared among multiple users. Each user, and in one or moreembodiments each eye of each user, may see a different image from thedisplay. In one or more embodiments, the system may track the positionand orientation of the glasses, for example so that the image or imagesviewed by each user reflects the user's viewpoint. Tracking may forexample use a camera that locates the glasses and determines theglasses' orientation and position. Calculation of orientation andposition may for example use image analysis based on known features ofthe glasses, such as the round shapes of lenses or frames, or otherdistinguishing features. In one or more embodiments calculation oforientation and position may analyze features of the user's face or ofany other objects on or near the user's head, instead of or in additionto analyzing features of the glasses.

One or more embodiments of the invention relate to eyewear with lensesthat selectively permit images to be received by one eye but not theother. The eyewear or glasses may be employed in stereoscopic displaysystems and also dual view systems where one observer sees a differentimage than another. Prior art methods include shutter glasses, passivelypolarized glasses, or anaglyph glasses. One or more embodiments of theinvention may combine prior methods of stereoscopic or dual view glassesin order to increase the number of images that may be discriminated bythe lenses. This allows dual view stereoscopic imagery as well asmultiple views beyond mere dual view.

For example, one or more embodiments may incorporate two or moreeyeglasses, each worn by a corresponding user. Each of these eyeglassesmay contain a pair of lenses. The users may view a shared display. Thedisplay may emit multiple images for each frame of a video stream. Theglasses and the display may be configured so that only selected imagesfrom these multiple images reach each user, or each eye of each user.This may be achieved by associating multiple characteristics with eachimage emitted from the display, and by configuring the lenses of theeyeglasses to select images having a specific combination ofcharacteristics. For example, lenses may include two or more selectivebarriers, each of which only allows images to pass through the barrierif the image characteristics match the characteristic associated withthe barrier. Characteristics may include for example, withoutlimitation, a range of colors, a light polarization, or a time sliceduring which an image is emitted from the shared display. Thecorresponding barriers associated with a lens may include for example,without limitation, an anaglyph filter that transmits only light in aspecific range of colors, a polarized filter that transmits only lighthaving a specific polarization, and a shutter lens that is open totransmit light only during a specific time slice.

One or more embodiments may use glasses that combine two or moreselective barriers for each lens. For example, these two barriers mayinclude combinations such as an anaglyph filter combined with apolarized filter, an anaglyph filter combined with a shutter lens, or apolarized filter combined with a shutter lens. One or more embodimentsuse lenses with three (or more) barriers, such as barriers that includean anaglyph filter, a polarized filter, and a shutter lens.

Glasses with various combinations of selective barriers may be used toprovide different views to each user. For example, one or moreembodiments may use four or more eyeglasses, and may provide differentviews to each of the eyeglasses by varying the barriers associated witheach eyeglasses. In one or more embodiments, the left lens of one ormore of the eyeglasses may have different selective barriers from theright lens, thereby providing stereo vision with different left eyeimages and right eye images selected from a stereoscopic image pair.

An illustrative embodiment that uses combinations of selective barriersis a quad view system with a single shared monitor viewed by two (ormore) users. The monitor may for example emit four (or more) images,each with a unique combination of polarization and time slice. Two (ormore) users may view the monitor with glasses that have lenses to selectone of the four images for each eye. Two users may therefore each view adifferent stereoscopic image from the same monitor. For example, lensesmay have a polarization filter at or near the front side of the lens,and a shutter filter at or near the back side. One or more embodimentsmay use circular polarization, which for example provides viewing thatis not distorted when a user rotates his or her head. For circularlypolarized and time sliced images, an illustrative lens may use a waveretarder in the polarization filter to select the correct circularlypolarized image (and block other polarizations) and convert it to alinear polarization. The shutter filter may for example use liquidcrystals to selectively twist the linear polarization to a differentorientation during the correct time slice, followed by a final linearpolarizer to block images without the appropriate linear orientation.

One or more embodiments of the invention may allow use of polarizedlenses with the displays employed in automobiles and aircraft, forexample by adding a wave plate to the displays. A wave plate may forexample convert linearly polarized light emitted by these displays intocircularly polarized or elliptically polarized light, thereby allowinguse of polarized lenses to view the displays.

One or more embodiments of the invention may incorporate tracking ofglasses. This tracking may for example use an image analysis system toanalyze images of the glasses, which may for example be captured by acamera. The image analysis system may calculate the position andorientation of the glasses from the camera images, or from other sensordata. The system may include at least one sensor. The sensor or sensorsmay be coupled to the display device in a known position and orientationwith reference to the display device. The sensors may be employed at afixed location in relation to the display but need not be co-locatedwith the display. The system may include object recognition softwaretechniques also known as computer vision. At least one processing unitmay be employed to process the data. At least one of the sensors maydetect light received from any portion of the glasses.

In one or more embodiments, the sensors may include a stereo camera, andthe system may analyze stereo images captured by the stereo camera todetermine the distance to a tracked object. In one or more embodiments,the sensors may include a plenoptic camera, and the system may analyzelight field images captured by the plenoptic camera to determine thedistance to a tracked object. One or more embodiments may incorporatedistance sensors on the eyeglasses that measure the distance from theglasses to the shared display (or to any other reference point orreference object), and transmit this measured distance to the imageanalysis system.

Glasses may include a surface including at least one circular geometricshape. The geometric object may be a circle or globe. The geometricobject may be other objects besides the circle and is not intended to belimited in this regard. Multiple blobs or objects may also be employed.Patterns of light and dark contrast may be reversed between pairs ofglasses or between lenses on the same pair of glasses. Differentglasses, different lenses of the same glasses, or both, may havedistinctive features so that each can be recognized and tracked based onits visual appearance.

The system may include a processor coupled to at least one of thesensors. The processor may assess, during use, a referenced position ofthe geometric object. It may calculate the three-dimensional positionand orientation of the object from its two-dimensional projection. Inaddition, when the geometric object is a circle or globe, the processormay assess the length of a major and/or minor axis of the geometricobject. The length of the major or minor axis may be assessed by numbersof pixels in the field of view or by the angle in relation to the fieldof view. The length of major and/or minor axes may be used to aid inassessing the location and/or orientation of the geometric object inrelation to the camera. Furthermore, the major and minor axes location,direction and size when taken in combination may be used to assess thelocation of the eyes.

This is an additional improvement over prior art, which merely discussestracking the surface that is tracked and fails to account for the factthat the eyes may be a centimeter or more behind the surface in variousangular directions.

The processor may determine the position of the geometric objects or theeyes behind them with reference to the display. The image processor maygenerate, during use, one or more images corresponding to a viewpointrelated to the position/orientation of the glasses or of the eyes withrespect to the display. In one or more embodiments, the image processormay generate a stereoscopic pair of images corresponding to the left andright lens positions and orientations or to the positions andorientations of the left and right eyes.

The surface of the eyeglasses may include at least one reflector. Forexample, without limitation, the rim of the lenses may include areflective material. The surface may also include an electroluminescentpanel. Any portion of the glasses may emit light; for example the rimsof the lenses may emit light. The light received from the geometricobject on the surface may be reflected or emitted light when thegeometric object(s) of the surface are detected substantially inproximity to the display device during use. The emitted light may comefrom an electroluminescent panel. One or more embodiments may include alight source that illuminates the eyeglasses. The light source may befor example, without limitation, an infrared or ultraviolet lightsource; the reflective portions of the glasses may reflect the infraredor ultraviolet light received from the light source. In one or moreembodiments, the light source may be coupled to a sensor that detectsthe level of ambient light, and it may be configured to turn on onlywhen this level of ambient light is below a threshold.

In one or more embodiments, the shared display may transmit multipleimages corresponding to the tracked positions and orientations ofmultiple glasses, or to the tracked positions and orientations of theleft and right eyes of the users (for stereo images). The images may beconfigured so that each of the eyeglasses (or each lens) receives onlythe images corresponding to the viewpoint associated with the trackedposition and orientation of the corresponding glasses, lens, or eye.

The system may include additional tracking blobs located on the glassesin a different plane than the original circular or noncircular pattern.In this way tilt, roll, and yaw information may be obtained. From thetilt, yaw, and roll information a 3D pointer beam may be created usingstereoscopic images from the display panel. Thus, the glasses themselvesmay be used as a pointing device. When the created pointer beamintersects with an object, interaction may be made to occur.Additionally, the pointer beam may be combined with other inputs from akeyboard, mouse, game pad, track ball, or other input device to enhanceand increase the interaction with 3D stereoscopic objects. For example,one button may be used to grab the object. Depending on context a buttonmay be used for firing a stereoscopic gun that hovers in front of theglasses. These are just a few examples and there are many morepossibilities, so the examples are not intended to be limiting in anyway.

A beam may be created in 3D from the central portion of the glasses thatextends in a perpendicular direction from the glasses. The beam itselfmay be created from the display image in such a way as to appear to theuser as though the origination point is approximately the center of thefront plate of the glasses frame. Additional lines and a pointer spotmay also be drawn in 3D from the image display device. These additionallines may help the user to get a good feel for the direction of thepointer. The distance from the display of the glasses previouslydescribed in the instant invention may be found using the methodsdescribed herein. For the pointer to operate correctly in addition tothe distance, the angular tilt of the glasses must be known. Forpurposes of discussion there are three angles of rotation for the headand since the glasses are attached to the head, for the glassesthemselves. If we consider the Z-axis to be extending forward from thehead, the Y-axis as up and down, and the X-axis as to the left and rightof the head then we can describe the rotations of the head as follows:There is rotation of the head from shoulder to shoulder about theZ-axis. This may be referred to as “roll.” There is rotation of the headfrom left to right, as one would see when an individual is indicating“no” by head movement. This is rotation about the Y-axis and may also bereferred to as “yaw.” Finally, there is a nodding up and down of thehead as a “yes” response might look. This is rotation about the X-axisand may also be referred to as “tilt.” To enable the tilt, yaw and rollaxis to be determined additional tracking points out of the plane of theoriginal tracked circles (or other shapes) may be employed. By comparingthe midpoints between these additional tracked objects with thecoordinates of the circles information regarding the tilt, roll, and yawmay be deduced. By employing the pitch, roll, and yaw data combined withdistance a line from the center and perpendicular to the front plane ofthe glasses may be constructed using the 3D imaging display panel thatis visible to the user. When this line drawn from the plane of theglasses intersects with the location of a 3D stereoscopic image drawnfrom the same display interaction may be made to occur. In a similarfashion, there may be placed a drawn 3D object in front of the glasses.This 3D object may be made to look like and represent many differentobjects, including for example, without limitation, an airplane, a birdor other flying animal or insect, a gun, and a projectile firing object.This list is meant to give an idea of the possibilities, but is not tomeant as limiting the object that may be presented in front of theglasses.

In addition, the created object in front of the glasses may be used as acalibration tool. By using keyboard commands, adjustments to thelocation of the object may be made thus providing feedback to thecomputing device regarding the location of the 3D object and hence theglasses. In addition, the 3D object created in relation to the glassesmay be compared with the location of one or more 3D objects created inrelation to the display panel to enhance calibration.

Finally, it should be noted that in multi-view systems any stereoscopicobject created by the processor for display in 3D may be seen at thecorrect position based upon the individual user's point of view. Forexample, a beam projecting outward from the front pane of a first user'sglasses would be drawn differently for the second user. In this manner,the second viewer sees the 3D beam in the same location, projectingoutward in front of the first user's glasses as well. This helps in ateaching environment when the beam is used as a pointer.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features and advantages of the inventionwill be more apparent from the following more particular descriptionthereof, presented in conjunction with the following drawings wherein:

FIG. 1 is a schematic diagram illustrating prior art shutter glasses.

FIG. 2 is a schematic diagram illustrating prior art passively polarizedglasses and also prior art anaglyph glasses.

FIG. 3 is a schematic diagram illustrating a wave plate.

FIG. 4 is a schematic diagram illustrating glasses employing bothanaglyph and passively polarized lenses.

FIG. 5 is a schematic diagram illustrating glasses employing bothanaglyph and shutter lenses.

FIG. 6 is a schematic diagram illustrating glasses employing bothpassively polarized and shutter lenses.

FIG. 7 is a schematic diagram illustrating glasses employing anaglyph,passively polarized, and shutter lenses.

FIG. 8 is a schematic diagram illustrating glasses employing bothpassively polarized and shutter lenses.

FIGS. 9A and 9B are schematic diagrams illustrating glasses employingboth passively polarized and shutter lenses.

FIG. 10 is a schematic diagram illustrating prior art displays in anautomobile that emit linearly polarized light.

FIG. 11 is a schematic diagram illustrating automobile displays thatemit circularly or elliptically polarized light.

FIG. 12 is a schematic diagram illustrating a display signal arrangementwhich uses both side by side and anaglyph methods in combination.

FIG. 13 is a schematic diagram illustrating a glasses arrangement fortwo-person stereo using side by side and anaglyph methods incombination. This may be used for two-person point of view stereo.

FIG. 14 is a schematic diagram illustrating a display combining side byside and anaglyph stereo methods to produce four unique image viewcapability.

FIG. 15 is a schematic diagram illustrating a display signal thatcombines side by side and top and bottom combination enabling fourviews. At least one embodiment of the method combines passive glassestechnology with shutter technology. Very fast shutter glasses technologymay also be used, but may be more expensive due to the shutter speedrequired.

FIG. 16 is a schematic diagram illustrating a signal that is split intofour views placed one above the other or placed side by side with eachother.

FIG. 17 shows embodiments of trackable glasses with circular lenses.

FIG. 18 shows how a circular object may appear when viewed fromdifferent angles.

FIG. 19 shows an embodiment of a system that tracks a user wearingglasses with circular lenses.

FIG. 20 illustrates an embodiment that tracks glasses using twoinexpensive cameras separated by a known distance.

FIG. 21 shows an embodiment of trackable glasses with different featureson left and right lenses.

FIG. 22 shows an embodiment of a tracking system that uses a light toilluminate the lenses.

FIG. 23 shows an embodiment that employs circular rings of differentcontrast around the lenses.

FIG. 24 shows illustrative embodiments of trackable lenses of othergeometric shapes.

FIG. 25 illustrates tracking a circular lens through various angles asseen by a camera.

FIG. 26 illustrates how an embodiment of the system calculates the x-y-zlocation of the lenses.

FIG. 27 illustrates another method for determining the z-distancebetween trackable glasses and a display screen that uses distancemeasuring equipment attached to the glasses.

FIG. 28 shows an illustrative flowchart for tracking glasses.

FIG. 29 shows an illustrative template for paper glasses employing acircular shape for tracking.

FIG. 30 shows an embodiment of a calibration tool.

FIG. 31 illustrates a user interface for 3D stereoscopic sculpting usingtracking glasses.

FIG. 32 shows illustrative 3D sculpting using a virtual pottery wheel.

FIG. 33 illustrates display of different 3D images to two differentusers using multi-view, trackable glasses.

FIG. 34 shows an embodiment where the user interacts with a 3D object ata distance.

FIG. 35 shows an embodiment of a laptop or folding computer that may beused to create and manipulate 3D stereoscopic images.

FIG. 36 illustrates an embodiment where the circles used for trackingare attached to a hat or other headgear.

FIG. 37 illustrates an embodiment that uses a circle as a generaldistance measuring device.

FIG. 38 shows a circular shape that may for example be printed from acomputer image file for distance measurement.

FIG. 39 shows an embodiment that uses two circles as a general distancemeasuring device.

FIG. 40 shows an illustrative flow chart of a process for determiningdistance to a flat circular object.

FIG. 41 shows an embodiment with multiple glasses having differentfeatures to support tracking in a multi-user viewing environment.

FIG. 42 shows an embodiment of glasses with added tracking dots or blobsin the four corners of the front facing surface.

FIG. 43 illustrates possible rotational movements of the user's headwhen wearing tracking glasses.

FIGS. 44A, 44B, and 44C illustrate a method for using the tracked dotstogether with tracked circular objects to calculate pitch, roll, andyaw.

FIG. 45 illustrates an embodiment of a pointing system that uses trackedglasses to control a 3D pointer.

FIG. 46 illustrates an embodiment of a quad view system with viewingglasses that combine a polarizing filter with a liquid crystal shutterto generate four different possible views of a single monitor, therebysupporting for example dual stereoscopic views.

FIG. 47 shows a flowchart of the operation of an embodiment of the quadview system.

FIG. 48 shows an embodiment of the system that determines the outline ofa geometric shape by detecting and analyzing a pattern of dots in thelenses of viewing glasses, in order to determine the 3D location andorientation of the lenses.

DETAILED DESCRIPTION OF THE INVENTION

A quad view display system will now be described. In the followingexemplary description, numerous specific details are set forth in orderto provide a more thorough understanding of embodiments of theinvention. It will be apparent, however, to an artisan of ordinary skillthat the present invention may be practiced without incorporating allaspects of the specific details described herein. In other instances,specific features, quantities, or measurements well known to those ofordinary skill in the art have not been described in detail so as not toobscure the invention. Readers should note that although examples of theinvention are set forth herein, the claims, and the full scope of anyequivalents, are what define the metes and bounds of the invention.

Glossary

Unless defined otherwise, all technical and scientific terms used hereinhave the same meaning as commonly understood by one of ordinary skill inthe art.

The term “geometric object” as used herein generally refers to asensor-detected signal (e.g., reflection) that may be recognized byobject recognition software. The geometric object may be circular, butis not limited to a circularly shaped object.

The term “connected” as used herein generally refers to pieces which maybe joined or linked together.

The term “coupled” as used herein generally refers to pieces which maybe used operatively with each other, or joined or linked together, withor without one or more intervening members.

The term “directly” as used herein generally refers to one structure inphysical contact with another structure, or, when used in reference to aprocedure, means that one process effects another process or structurewithout the involvement of an intermediate step or component. The term“emitter” as used herein generally refers to a device that projects asignal (e.g., light, infrared light, etc.). The emitter may be active(i.e., the signal originates from the emitter) or the emitter may bepassive (i.e., the signal originates from somewhere other than theemitter and is, for example, reflected off the emitter).

The term “eyepoint” as used herein generally refers to the physicalviewpoint of a single eye or a pair of eyes. A viewpoint above maycorrespond to the eyepoint of a person. For example, a person's eyepointin the physical space has a corresponding viewpoint in the virtualspace.

The term “head tracking” as used herein generally refers to tracking theposition/orientation of the head in a volume. This allows the user to“look around” a virtual reality environment simply by moving the headwithout the need for a separate controller to change the angle of theimagery.

The term “position/orientation” as used herein generally refers toposition/orientation in at least 2 degrees of freedom (e.g., onedimension position and one dimension orientation . . . X, rotation).Position/orientation may be relative or absolute, as desired.Position/orientation may also include yaw, pitch, and roll information,e.g., when defining the orientation of a viewpoint.

The term “referenced” as used herein generally refers to a known and/orcalculated (e.g., to a processor) precise position/orientation relationof a first object(s) (e.g., a sensor) to a second object(s) (e.g., adisplay device). The relationship, in some embodiments, may bepredetermined in that the relationship is fixed (e.g. physically fixedas in using precision spatial mounts) such that the relationship is notadjustable after initial assembly (e.g., wherein the first object andthe second object are assembled together as part of a single device).The relationship, in some embodiments, may be determined, during use,through a process (e. g., an initialization process, which may include acalibration and/or measurement process) which determines a precisespatial position/orientation relation of the first object(s) (e. g., asensor) to the second object(s) (e.g., a display device).

The term “sensor” as used herein generally refers to a converter thatmeasures a physical quantity and converts it into a signal which can beread by an observer or by an instrument. Sensors may include cameras,photo detectors, electronic sensors, CMOS or CCD sensors, etc.

The term “viewpoint” as used herein generally has the full extent of itsordinary meaning in the field of computer graphics/cameras. For example,the term “viewpoint” may refer to a single point of view (e. g., for asingle eye) or a pair of points of view (e.g., for a pair of eyes).Thus, viewpoint may refer to the view from a single eye, or may refer tothe two points of view from a pair of eyes. A “single viewpoint” mayspecify that the viewpoint refers to only a single point of view and a“paired viewpoint” or “stereoscopic viewpoint” may specify that theviewpoint refers to two points of view (and not one). Where theviewpoint is that of a user, this viewpoint may be referred to as aneyepoint. The term “virtual viewpoint” refers to a viewpoint from withina virtual representation or 3D scene.

Glasses that Provide Multiple Views

With reference now to FIG. 1 of the drawings, there is shown anillustration of prior art glasses (item 108) which employs liquidcrystal shutters. Each shutter (items 109 and 110) opens and closes insynchronization with first and second image presentation on a displaypanel to ensure correlation with the correct eye.

With reference now to FIG. 2 of the drawings, there is shown anillustration of prior art glasses (item 108) which employ passivelypolarized lenses (items 119 and 120). Said passively polarized lensesare coordinated with circularly or elliptically presented images on adisplay panel. This ensures correlation with the correct eye. The thirdmethod of prior art in this field employs anaglyph or colored lenses(items 129 and 130) designed to filter light based upon color. Again,this is used to ensure correlation with the correct eye.

With reference now to FIG. 3 of the drawings, there is shown anillustration of a wave plate (item 202). Light coming from one side ofthe wave plate is circularly polarized and the light coming from otherside is linearly polarized. Examples of this would be quarter or halfwave plates. The degree of circularity is determined by the thickness ofthe polarizing media and other thicknesses may produce similar results.

With reference now to FIG. 4 of the drawings, there is shown anillustration of glasses (item 118) that combine the two methods ofanaglyph lenses (items 129 and 130) and passively polarized lenses(items 119 and 120). The order of lenses from front to back isinterchangeable as is the direction of polarization. Glasses may becustomized to provide for stereoscopy, dual view stereoscopy, quad viewimagery or one stereoscopic and two non-stereoscopic views.

With reference now to FIG. 5 of the drawings, there is shown anillustration of glasses (item 118) that combine the two methods ofanaglyph lenses (items 129 and 130) and shutter lenses (items 109 and110). The order of lenses from front to back is interchangeable.

With reference now to FIG. 6 of the drawings, there is shown anillustration of glasses (item 118) that combine the two methods ofpassively polarized lenses (items 119 and 120) and shutter lenses (items109 and 110). The order of lenses from front to back is notinterchangeable as the passively polarized lenses must be closest to thedisplay panel.

With reference now to FIG. 7 of the drawings, there is shown anillustration of glasses (item 128) that combine three methods: shutterlenses (item 110), passively polarized lenses (item 120) and anaglyphlenses (item 130). The order of lenses from front to back isinterchangeable with the exception that the passively polarized lensesmust be nearer to the display panel than the shutter lenses.

With reference now to FIG. 8 of the drawings, there is shown anillustration of glasses (item 108) that combine passively polarizedlenses and shutter lenses in a combination that could be used for quadview of four different images by four users. By opening and closing theshutters (items 804 and 806) together and shutters (items 814 and 816)together in opposition of pairs we obtain dual view. When this iscombined with paired passively polarized lenses we obtain quad view. Inthis way four people each with their own set of individualized glasseswould receive a different view. This would be especially useful in gameplaying. Many games such as bridge require four players with each havingprivate knowledge.

With reference now to FIGS. 9A and 9B of the drawings, there is shownillustrations of glasses (items 802, 812, 852, and 862) that combinepassively polarized lenses and shutter lenses in different combinationsfor different effects.

For all of the different glasses in the present invention it isunderstood that by varying the combination of lenses multiple imagery orstereoscopic effects may be produced and the different combinations arelimited only by one's imagination. All combinations of anaglyph, shutterand passively polarized lenses are considered to be within the scope ofthis invention. Some combinations will enable more stereoscopic imagesto be seen while others may be employed which afford less stereoscopicimages and more non-stereoscopic images to be seen. In this way theplacement and types of lenses are flexible depending on the needs andwants of the user.

With reference now to FIG. 10 of the drawings, there is shown anillustration of prior art liquid crystal displays which are presentlyemployed in automobiles, airplanes, and transportation in general.Liquid crystal displays employ a front surface of linear polarizingmaterial to selectively block light based upon the twist ofpolarization; therefore, the light is emitted with linear polarization.The interior of an automobile is shown for illustration with item 170representing the windscreen, item 162 the steering wheel, and item 160the glove box. Items 150 and 152 illustrate liquid crystal displays thatemit linearly polarized light. This polarized light may or may not passthrough the polarized lenses (items 139 and 140) of a driver's glasses(item 108) depending on the angular orientation. Prior art displaypanels in automobiles and airplanes make use of liquid crystal displaytechnology or other technology which employs a linear polarizer on thefront or viewing panel. This is used to block unwanted light fromreaching the viewer. However, this technology makes it difficult forusers of linearly polarized glasses to see the display properly. Oneresult of this is commercial pilots do not make use of polarized glasseswhen flying aircraft. This presents a problem for people who usepolarized glasses. As the plane of polarization from these glasses isrotated with respect to the displays various amounts of the linearpolarized light will reach the eye. It is for this reason aviatorsunglasses are of the non-polarized variety. Polarized lenses are moreeffective at reducing glare; however, they interfere with the displaypolarization as mentioned.

With reference now to FIG. 11 of the drawings, there is shown anillustration of an automobile employing a wave plate for the frontsurface of its displays (items 150 and 152). This wave plate may be theonly polarized front surface as one side accepts linearly polarizedlight while emitting circularly or elliptically polarized light from theother surface. Alternatively, a thin sheet of wave plate may be appliedover an existing display to change the linearly polarized light intocircular or elliptically polarized light.

The light leaving the display panel is now circularly or ellipticallypolarized and thus passes through linearly polarized lenses of eyewearregardless of the angle. In this way aviators, automobile drivers, etc.,may now use glare-reducing lenses in their eyewear without fear oflosing visual sight of their instrumentation. This makes for safertransportation.

There are other display technologies that employ liquid crystal displaysand emit linearly polarized light. Said displays may be converted toemit elliptically or circularly polarized light by the methods describedin the present invention. Hence, all screens for automotive, aviation ortransportation use, which employ a front surface of circularly orelliptically polarized light or light other than linear polarized light,shall be considered within the scope of the present invention.

FIG. 12 is an illustration of an embodiment of the display mode whichemploys two methods of stereo mode. The split screen format is currentlyin use in most 3D displays of the present time. In this display mode afirst and second images are displayed on left and right halves of asignal sent to the display unit. The display unit usually will have a 3Ddisplay mode which splits the halves and displays the first or secondimages in full screen width to the left and right eyes in a coordinatedmanner so as to produce a stereoscopic effect. There are two commonmeans for accomplishing this. The first employs shutter glasses wherethe first and second images are displayed with a time differential. Theleft and right shutter glasses open alternately in sequence so thecoordinated image reaches the proper eye. The second method employspassively polarized glasses where the left and right eyes have oppositepolarization. The first and second images are displayed with oppositepolarization so the coordinated image is able to pass through the properlens to achieve 3D stereo effect. A third method of employs parallaxbarriers to achieve auto stereoscopy.

In the embodiment illustrated in FIG. 12, the arrows (items 1202 and1204) are used to illustrate the differentiation of the left and righthalves of the split screen. The arrows can be considered to representimages which will be displayed to different eyes where thedifferentiating feature may be polarization (for use with passivelypolarized glasses), sequential time (for use with shutter glasses), orseparation using parallax barrier technology. In this illustration, eachhalf of the screen displays images (items 1206 and 1208 on the lefthalf, and items 1210 and 1212 on the right half) which are furtherdifferentiated by use of a color filter. In this embodiment anaglyph orcolored glasses that allow light from one color to pass while blockinganother color are used to further coordinate which image passes throughthe properly coordinated lens of the glasses. The colors illustrated arered and blue, but this is not the only possible color combination andany combination of color discrimination that allows passage of one colorthrough one lens while filtering the other may be employed.

Therefore, the eyewear may have lenses that discriminate using twomethods. The first method may employ shutter lenses, passively polarizedlenses or parallax barrier technology. The second method may employcolored lenses that are also referred to as anaglyph lenses.

FIG. 13 expands the illustration of FIG. 12 to include the eyewear(items 1320 and 1322). In this illustration, the arrows are used toillustrate coordination with the left and right images of the splitscreen. The red and blue colors illustrate coordination with the imagesdisplayed using red and blue color filters. In this embodiment fourdistinct images can reach four different eyes by use of two sets offilters. The lenses of the glasses in this embodiment have two stages. Apolarization or shutter stage (parallax barriers may also substitute forthis stage) and a colored or anaglyph stage. If the images are properlycoordinated for point of view then two unique stereoscopic images may beseen by two viewers. Furthermore, if the location of the glasses or headposition are tracked then two unique point of view (also referred to asPOV) stereoscopic images may be seen.

Such an embodiment has tremendous use. If the images are properlyconstructed, two individuals may see a 3D stereoscopic image whichexists in the same location in space with each of the two viewers seeingthe image as it would appear from their point of view. This enables anillustration where the first user may point to a 3D displayed object andthe second user sees the location on the object being pointed to fromhis own viewpoint. This would be useful to educators, doctors explainingan image to a patient or to anyone talking about one image to another.In addition, using POV techniques real world virtual objects may becreated that may be interacted with by more than one user. This may beemployed in gaming devices and methods. It may also be employed insimulators or simulations.

In FIG. 14 we have an illustration of an embodiment where four viewerseach see their own unique display image. In this embodiment, the imagepresented is not stereoscopic. However, in another embodimentcombinations of three or more stereoscopic methods may be employed toallow four or more users to each see unique 3D stereoscopic images. Forthe embodiment illustrated in FIG. 14 we have four pairs of glasses(items 1430, 1432, 1434, and 1422). Each pair of glasses is implementedso it sees either the first or second image presented by the splitscreen discriminator (shutter lenses, passively polarized lenses, etc.).In this embodiment, the color discriminator is now the same for eachlens of a pair of glasses. In this way, both of the lenses in each pairof glasses only allow passage of one of the four images produced. So,each individual user sees the same one of four images in each eye. Thisallows gaming devices where four players can play and none see theothers' information. An example, but not the only use would be a game ofbridge. The glasses of any of the embodiments may be further enhanced byuse of side blocking devices so users playing games can't look aroundthe sides of the glasses.

The method of shutter glasses to display more than two images is limitedby the speed at which the glasses can cycle and also by the fact thatthere is a period before and after opening when nothing is displayed toavoid ghosting. Hence shutter technology by itself is of limited usebeyond two sequential images. In another embodiment of the instantinvention the method of shutter glasses is combined with the method ofpassively polarized glasses. This requires a display that employs bothmethods. Such a display requires a means for four images to be sent anddecoded. At present the side by side method where each side is expandedto full when displayed in 3D stereo is employed. FIG. 15 is anillustration of a method of compressing four images into one frame. Item1500 shows the frame split into quarters with each quarter displaying animage. The four images are then displayed full screen with each onehaving two methods applied. The system may for example combine themethods of shutter glasses and passively polarized glasses or maycombine one of these methods with parallax barriers. In anotherembodiment, each of the four images has the same ratio of length towidth as the full screen expansion. Thus, it is compressed by the sameratio in both length and width. FIG. 16 illustrates two othercompression methods for four images. Item 1610 is an illustration offour images one placed above the other. Item 1612 is an illustration offour images placed side by side four times.

In summary, the instant invention improves upon the prior art byproviding for multiple views of a stereoscopic display panel withoutrelying exclusively on the speed of shutter technology. Secondly, theinvention provides displays that work well with polarized glasses,thereby enhancing transportation safety.

FIG. 46 shows an illustrative embodiment of a system that supports fourdifferent views from a single monitor. Using this quad view system forexample it is possible for two viewers to be sent different stereoscopicimages and this may be applied to point-of view stereoscopic devices ormethods. Additionally, four separate 2D images may be presented to fourusers. This would have value for example as a four-person gaming device.Games such as bridge and scrabble may be played amongst four playerswith none the wiser as to the other players' cards or pieces.Additionally, in one or more embodiments one stereoscopic view may becombined with up to two two-dimensional views.

A quad view system may for example include a viewing monitor, acomputing processor, memory, and wave retarders. The viewing monitor mayhave a strobe, a timing coordination mechanism, wave retarder plates,and a means to coordinate timing with the viewing glasses of the users.

Existing 3D stereoscopic television/glasses systems known in the art useprimarily use one of two methods: time slice methods in which images aredisplayed sequentially in time and glasses have shutters that open andclose in coordination, and passive polarization methods that employ rowsor patterns of pixels that produce light patterns of opposite ellipticalor circular polarization. In the time slice method liquid crystalshutter glasses are employed. The light passes through a front plate andis given a linear polarization in a direction coordinated so that it maybe rotated or not rotated electronically by liquid crystals to passthrough a linear polarizer on the further side in order to reach theobservers' eyes.

One or more embodiments of the quad view system, such as the systemillustrated in FIG. 46, may combine a time slice monitor 4601 with apattern of wave retarders to produce circularly or ellipticallypolarized light in patterns. The patterns may be made to be parallelrows. For example, the monitor 4601 may output light 4602 that iscircularly polarized in a clockwise orientation, and light 4603 that iscircularly polarized in a counterclockwise orientation. The polarizationpattern of emitted light, together with a time slice during which thelight is emitted, may define a unique combination of one of fouroutputs. Each of these four outputs may for example show a differentview of a scene. In particular, in one or more embodiments two viewersmay each view a unique stereoscopic image of a scene, with differentleft and right images for each viewer (or four views in total).

Glasses 4610 used by a viewer to view the scene may for example combinetime slicing filters with a circular or linear retarder on or near thefront surface. For example, the right eye lens 4611 of the glasses 4610may have several layers that combine the selection of the correctpolarization with the selection of the correct time slice. By usingdifferent polarization or time slice selections for the left and rightlenses, and for different glasses for another viewer, the four viewsneeded for dual stereoscopic images may be generated. In the embodimentshown in FIG. 46, a wave retarder or polarizer 4613 is placed at or nearthe front of the lens 4611 (where “front” is closest to the monitor andfurthest from the eye). Different polarizers on the front of the leftand right lenses will cause a different direction of circularlypolarized light to pass through the left and right polarizers. The frontretarders of the glasses may be configured so that after passing throughthe light is now linearly polarized, so it may be twisted as desiredelectronically by the liquid crystal material of the lenses that providethe time slice filters. For example, circularly polarized light 4612passes through wave retarder 4613, and is converted to linearlypolarized light 4614. The subsequent layers 4615, 4616, 4617, and 4618provide a shutter feature to either pass the light 4614 through to theviewing eye 4620, or to block it. Specifically, the liquid crystals 4616may twist to rotate the polarization 4614 to match the final linearpolarizing filter 4618, thereby passing the light through. If thevoltage on the liquid crystals is modified, the twisting may not occuror may occur differently, in which case the original polarization 4614will be blocked by filter 4618. Alternating the twisting and untwistingof the crystals thereby creates a time-sliced shutter filter.

Monitor 4601 may output images that are coordinated with glasses 4610and with a second pair of glasses. For example during time slice one afirst image may appear on odd lines 1, 3, 5, . . . of monitor 4601, andimage two may appear on even lines 2, 4, 6, . . . of monitor 4601. Theseeven and odd lines may be output with different polarizations. Duringtime slice two a third image may appear on odd lines 1, 3, 5, . . . andimage number 4 may appear on even lines 2, 4, 6 . . . This process maythen be repeated for subsequent images.

The glasses may coordinate with the monitor so that the proper imagereaches the correct eye. For example, one set of glasses may have bothshutters closed while the other has both shutters open. Or they may becontrolled so the left eye shutter of glasses pair number one is closedat the same time slice as the right eye shutter of glasses number two isalso closed. The coordination of images with correct lens and eye may becontrolled by both passive polarization and time slice working inconjunction.

FIG. 47 shows an illustrative flowchart of steps that may be used in oneor more embodiments to provide four (or more) different views. In step4701, four different display outputs are generated, each with a uniquecombination of polarization and time slice. In step 4702, one of theseoutputs enters the front of a glasses lens, which selects for one typeof polarization and blocks the other type. In steps 4703, the linearlypolarized light emitted from the front wave retarder of the lens istransmitted to the front of a shutter system. An illustrative shuttersystem uses rotating crystals to either rotate the linear polarizationor leave it unchanged. In step 4704, electricity applied to the liquidcrystals causes them to twist, thereby rotating the polarization. Instep 4705, the linearly polarized (and possibly rotated) light istransmitted to a final linear polarizer, which either transmits orblocks the light. In step 4706, images are coordinated by polarizationand by time slice to match the appropriate lens of the appropriateglasses.

Tracking of Glasses

This disclosure also describes systems and methods for, in someembodiments, tracking a head of a user relative to a display system;this may include a tracking device which couples, during use, to a headof a user. In some embodiments, the tracking device may include eyewear,headwear, arm wear, hand wear, object cover and/or another device thatis to correlate to some object to be tracked. The tracking device mayinclude a first side, a second side, and at least one geometric objectemitter. The second side may be opposite the first side. The second sidemay be directed, during use, towards the head of the user. The at leastone object emitter may be positioned on the first side of the trackingdevice. In one or more embodiments, this emitter may be a reflectorwhich reflects ambient light, IR or UV light.

One or more embodiments of the system may track any feature or features,including but not limited to features of eyewear. Tracked features mayfor example include facial features such as eyes, ears, noses, ormouths. Tracked features may include for example any 2D objects locatednear or attached to the head (including but not limited to the use offacial features). Tracking may be based for example on locating anyfeature or features in one or more 2D images (such as for example imagescaptured by one or more cameras), and calculating the 3D position andorientation of a user's head or any parts thereof (such as eyes) fromthe 2D images of these features. Determining the 3D position andorientation of the user's head or of any other objects may be based forexample, without limitation, on the size, orientation, or shape of thefeatures in the 2D images, or on the separation between features in the2D images.

FIG. 17 shows illustrative embodiments of inexpensive eyewear withcircular lenses. Embodiments of glasses and goggles are shown, but thelenses may be mounted in other configurations such as attached to a hator headband. As FIG. 17 shows, the eyewear with circular lenses can bestylish and there are many frames that support circular lenses. In oneor more embodiments, glasses or goggles with sideways shieldingmaterials may be employed to prevent gamers from looking outside theedges of the glasses, preserving fair play.

There has lately been great progress in the field of computer vision.These advances make it possible using sensors or cameras to identifyobjects in the field of view of the sensors or cameras. Complex objectssuch as faces may be identified and facial features may be determined.The task of identifying basic geometric shapes is well within thecapabilities of modern computer vision software. One such popularcomputer vision software is called OpenCV and there is much literatureregarding how to use it. One reference is the book “Learning OpenCV,Computer Vision with the OpenCV Library,” by O'Reilly. The websiteopencv.org has additional online documentation and tutorials.

An online source for identifying circles using inexpensive camerascoupled with computer software can be found at:http://www.pyimagesearch.com/2014/07/21/detecting-circles-images-using-opencv-hough-circles/.Additionally, methods for identifying ovals in images can be found at:http://scikit-image.org/docs/dev/auto_examples/plot_circular_elliptical_hough_transform.html.Information on faster ellipse detection techniques is available on theInternet. These are just examples of methods for identifying circlesand/or ovals and other methods may be used to accomplish the sameobjective without deviating from the scope of the present disclosure.

FIG. 18 shows how a circular object may appear when viewed fromdifferent angles. It will generally have an oval shape. Consider a coinin front of the eye. When viewed front on the coin presents a circularshape. As the coin is rotated about an axis the eye sees an ovularshape. When viewed from on edge the coin appears as a line segmentwould. In this way, a circle generates an oval when viewed fromoff-center. An oval has a major and minor axis. The center will be theintersection of the major and minor axis. The major axis will always beequal to the diameter of the corresponding circular shape that generatedthe oval. As an example, consider a coin too large to pass through thetop of a soda bottle that will not pass through no matter the angle thecoin is paced in relation to the bottle opening.

At the time of this invention shape sensing technology is an establishedfield. Someone skilled in the art is capable of finding circles,ellipses and other geometric shapes in a scene. In addition, determiningthe major, minor axis and orientation of an ellipse within a scene arewell within the capabilities of someone skilled in the art of shapedetection. “OpenCV” or Open Computer Vision is the current softwarelanguage employed for this purpose. Additionally, multiple scenes may becaptured by computing devices. One set of software instructions for thisis called “DirectShow” or “DShow” for short. So sequential scenes may becaptured and analyzed in sequence to provide updated positions of thedata for the lenses and by extension the viewpoint locations of the eyesof the observer.

There is always a long axis equal to the diameter of the coin that willprevent the coin from entering the bottle. It is this principal whichallows us to calculate the distance of the circle from the camera. Wecan measure the major axis, which is equal to the diameter of thecircle. The length of this in proportion to the viewing angle of thecamera is then compared with the known length of the diameter and theproportion to the viewing angle at a known distance from the camera. Wedo a mathematical comparison of the viewed length at an unknown distancewith the known length at a known length.

To further clarify, a penny viewed at a given distance will always havea major axis of the same length no matter how it is rotated about anyaxis that passes through the center. The same holds true for anycircular shaped object. This is what makes this shape of value. If thediameter of the circle is known we can use the length of the major axisas viewed from the sensor combined with trigonometry to determine thecircle's distance from the sensor. Therefore, in some of our embodimentexamples we employ circular shapes to the lenses or to the lens frames.

In this way, we are able to compute the distance of the circular lens orlens frame from the camera regardless of the tilt or rotation of thecircle along any axis in relation to the camera. The key is the circularnature of the lens or lens frame which when seen from any angle presentsan ovular shape. It should be noted that a circle is an oval. A linesegment is also ovular and represents what would be seen if a circlewere viewed along its edge. The length of the major axis of the observedoval is the same length as the circle's diameter would be when viewedfrom the same distance.

The formula relating camera angle to circle diameter is given by:

Tan(a/2)=Diameter of Circle/(2*Distance to Object)

where “a” is the angle made by the camera with both ends of the Majoraxis.

If an assumption is made about the location of the eye in relation tothe circular lens then the major and minor axis taken together may beused to assess the location of the eye. One or more additional geometricobjects may assist in this assessment.

FIG. 19 shows an embodiment of the system or method in the presentinvention. A viewer wears glasses of circular lenses. These lensespresent ovals to the camera located on or in fixed location withrelation to a viewing screen. A computing device takes the informationand calculates the location of the viewpoint. Based on the viewpoint animage is created as would be seen from this viewpoint. 3D images may becreated for each eye; discriminating lenses then permit the correctimage to reach the proper eye. In this way, real world stereoscopicimages may be created. These images remain fixed in space within a smallmargin of error. Calculating the eyes' location behind the lenses andbasing the image creation on the approximation of the eyes' location mayfurther reduce this error.

It is also possible to use shape recognition technology to identify theshape of glasses frames without identifying circular features. In oneembodiment, the known size and shape of the glasses frames can becompared with the size of the captured image to compute the distance andlocation of the glasses lenses. However, the method of identifyingellipses uses less computer power and therefore enables more frames perminute to be displayed. In the future, as computing power increases themethod of identifying the glasses frames using computer vision maybecome an advantageous embodiment.

One of the potential issues with shape tracking is unwanted noise. Bythis is meant tracking of unwanted objects. If these incorrectly sensedobjects are mistakenly used as data for computing the viewpoint theimage will behave erratically. This will result in viewer displeasure.At least one embodiment of the instant invention may employ filters toremove these unwanted shapes. One means to accomplish this is by asmoothing algorithm similar to a “Kalman filter.” Bad data points may bethrown out. In at least one embodiment, the two circles or ovals desiredto be tracked are in close proximity to one another and share manytraits. The sizes of the tracked shapes are nearly the same. The anglesof the major and minor axis are aligned. The ratios of major to minoraxes will be the same. Because the rotation of the major axes can besensed, the ratio of the distance between centers of the lenses to thelengths of the major and minor axis may be computed using trigonometry.These facts and others may be used in at least one embodiment to filterout bad data. Other means to filter out bad data include, but are notlimited to distance filters, use of high contrast materials andfiltering out low contrast objects. Additionally, plenoptic or “lightfield” technology may be employed to sense the distance and compare itwith expected distance of the lenses from the camera. The plenopticcamera uses software to take several photos of a scene with differingfocal distances. It then uses a computing device to examine when objectsare in focus. It uses this data to calculate distance to the object.Employing some or all of these methods will ensure mostly good datapoints for the viewpoint are used. Therefore, the result the imagescreated based on the data will be smooth. This will increase viewingpleasure.

The camera may record visible light as well as UV or IR light. An IR orUV light source may be reflected off of the lens or lens frame of theeyewear. This can be accomplished by using IR or UV reflective materialsin the construction of the glasses but reflective tape, paint or otherreflective or retro reflective material may be used. In addition, theglasses may contain their own source of circular light around the rim.This light may be visible, IR, UV or other frequency.

Another embodiment employs a plenoptic or “light field” camera. Thesetypes of camera take multiple pictures of the same scene with differentfocal lengths. By examining when objects are in focus the plenopticcamera is able to determine depth. This depth as sensed by the plenopticcamera may be useful for computing the Z distance from the viewingscreen. It should be noted that all sensors or cameras described in theembodiments of this document are assumed to be in a fixed location inrelation to the viewing screen.

FIG. 20 shows an embodiment that uses two inexpensive cameras a knowndistance apart from each other. By triangulation, the location anddistance of the eyewear may be determined. The circular shaped lensesmake the eyeglasses easy to be tracked and eliminates the need formarkers blobs or other devices. Markers, blobs or other devices could beemployed as differentiators however. In other words, they could be usedto differentiate one lens from another in one pair of eyewear. Theycould also be used to differentiate between sets of eyewear in amulti-user environment. In addition, the circular shape allows the majorand minor axis of the lenses to be calculated. This results in animproved ability to locate the eyeball viewpoint behind the lens.

FIG. 21 shows one embodiment of a lens differentiator. One lens is madeof concentric circles that will be observed by the image tracker. Inthis way, a left and right lens may be determined for any orientation orrotation even upside down. In this way, a viewer can get the correctstereoscopic association from any viewing tilt angle. This method mayalso be used to discriminate between two or more pairs of glasses thusenabling multi-viewer systems.

Other geometric shapes or blobs may be placed at strategic locations ofthe eyewear to facilitate the differentiation of first and secondviewpoints. In additional geometric objects placed on the surface of theeyewear may be useful for systems with more than one viewer. Theseadditional objects may be used to discriminate between several users soeach viewer sees the correct stereoscopic image.

FIG. 22 shows an embodiment of the instant invention that employs alight (item 2202). The light is configured to shine on the lenses of theglasses. In low light or low contrast, it may be difficult for thesensor (item 2206) and tracking system to locate the outline of thecircular lenses. In these situations, a light may be employed to enhancethe contrast of the lenses of the glasses. In this manner, the light maybe employed to enhance the glasses tracking capabilities of the system.The glasses (item 2204) may be further enhanced by fluorescent or dayglow coloring along the circular rims. The rims may be painted or moldedfrom colored material. Additionally, a darker circle may be placedaround the rim inside of the fluorescent colored material. The colorwhite or any other light color may work in place of or with fluorescentcoloring. Additionally, there are materials which when charged by lightenergy will re-emit the light over a period of time. These materials arereferred to as “glow-in-the-dark.” Sometimes phosphorescent materialsare used to create these “glow-in-the-dark” materials and the technologyis well understood. These materials may be used with or without theenhancement of a lighting device.

In another embodiment, the light (item 2202) may be of UV or ultraviolethue. This enables it to illuminate materials of fluorescent,phosphorescent nature more brilliantly than surrounding materials notmade of fluorescent, or phosphorescent nature. Creating or painting thecircular rims of the eyewear with fluorescent or “glow-in-the-dark”materials makes them pop-out of the field of view of the sensor enablingthem to be tracked more efficiently. This is especially important in lowambient light conditions. Additionally, reflective or retro reflectivematerials may be used and in one or more embodiments, the light may beplaced near the sensing device. In one or more embodiments, infrared(IR) light and IR reflective materials may be employed instead of, or inaddition to, UV light and UV materials.

Furthermore, the light (item 2202) may be controlled automatically ormanually. A user may manually control the light by means of a physicalswitch or the light may be connected to the computing device (item2208). The means of connection may be wired or wireless and may includebut not be limited to USB, Firewire, or Bluetooth. In this case, theuser may control the light via a signal sent through the computer bymeans of a keyboard or mouse input to turn the light on or off asneeded. The computing device may also be able to detect the ambientlighting conditions from the sensor. When the ambient light is sensed tohave fallen below a certain threshold the light turns on illuminatingthe eyewear. A photoreceptive cell that allows electrical current toflow when it senses light may also be used as a switch to turn the lighton and off based on lighting conditions.

A computing device (item 2208) analyses the data obtained from thesensor to calculate a position of the eyewear. This data analysis isdiscussed in greater detail in another section.

One of the problems faced by geometrical shape tracking systems isfinding the shape among many patterns. Enhancing the contrast of thegeometric shape is a means to improve the computing system's recognitionof shapes within a frame or scene. FIG. 23 shows embodiments of thecircular rim of the lens areas of the instant invention. Circular ringsof different contrast may be employed around the lenses of the eyewear.A dark colored ring which divides a wider light colored ring into twoareas and a light-colored ring which divides a wider dark colored ringinto two areas are shown. This makes it easier to track the lenses andwhen the lenses are of another geometric shape such as a square, thesecontrasting patterns may be modified for non-circular lens tracking. Thecontrasting patterns may be adapted in this fashion to follow along therims of glasses of any shape.

FIG. 24 shows embodiments of eyewear of other geometric shapes that maybe sensed and tracked. The circular lens shape is an embodiment thatprovides the simplicity of converting the sensed data into real worldX-Y-Z coordinates. A circle is after all a special case of an oval. Acircle when rotated will always produce an oval and the distance can becomputed by comparing the sensed length of the major axis with thesensed length of the major axis at a known distance. Because this is notso for other shapes they require additional steps and more complicatedalgorithms to convert the sensed position data to real world X-Y-Z data.Other shapes however have the advantage of having sharp edges that canbe detected easily. So, there may be circumstances where other geometricshapes may be useful and this will be dependent on the system employedto sense the lenses and the computing device as well as the how the userwill use the device. If the observer does not tilt the head in relationto the screen then the square would produce good results without muchmodification to the algorithms.

In one or more embodiments, lenses or other headwear may have patternsof dots embedded in them to facilitate reconstruction of geometricshapes, such as circles for example. A potential advantage of using dotsand then reconstructing a geometric shape from the outline through thedots is that it may be more robust in the presence of blurred images dueto motion of the image during image capture.

FIG. 48 shows an illustrative embodiment that tracks dots andreconstructs a shape (in this case a circle) from the dots. In thisillustration the outline of a geometric shaped object (circle 4802) iscreated using dots (such as dot 4801). There may be three or more dotsused to create the outline. In this illustrative embodiment the dots areround, however they may be of any shape. The camera 4810 may capture animage of the dots and a computing device may analyze the dots capturedin an image to create a geometric object that contains the dots. In thisexample the computing device would create a circle from the pattern ofdots. The circle is then analyzed to determine location as is describedelsewhere in this document.

The pattern of dots (including illustrative dot 4801) may be placedaround the lenses 109 and 110 of a pair of glasses 108. In this way the3D location of the lenses and hence the eyes may be determined.

In this way the problems associated with blurring of images can beovercome. As the user moves his or her head, geometric objectsassociated with the lenses may become blurred. This makes their locationdifficult to be accurately determined by a computing device. However,dots, while blurred, may be more easily located in an image. Hence, dotsin the shape of a geometric object may be used in lieu of the actualgeometric image. A computing device may find the blurred dots in animage. It may then evaluate the locations of the blurred dots andsimulate the geometric object. In this way the problems associated withdeciphering a blurred image may be overcome.

The dots may be differentiated from the background for example byshading, or by color.

Another method that may be used in one or more embodiments to overcomeissues associated with image blurring is the use of a strobe light. Thelight from the strobe light is reflected off the object image and backto the camera. The strobe light may be placed near the camera anddirected towards the image to be captured by the camera. Selectivelyreflective materials may be used to define the image object. These mayinclude IR or UV reflecting paint or materials. The strobe light mayemploy infrared or ultraviolet light. The strobe light may besynchronized with the camera to produce short time slice images therebyreducing blurring. A filter may be placed over the camera lens toselectively view the correct frequency of light reflected from theimage.

Alternatively, in one or more embodiments the image itself may be madeof light emitting materials such as electroluminescent paint. This paintmay be made to illuminate on and off very fast using electricity. Thispaint may emit light in the IR or UV range. Filters may be employed toselectively view the correct frequency of light. This also may reducethe time slice of the image thereby reducing blurring. The method ofusing strobe lighting or electroluminescent paint may enable the use ofthe original geometric object.

FIG. 25 shows circles that have been turned through various angles asseen by a camera. In each case the distance from the camera remains thesame. The major axis remains the same length, as does the angle formedbetween the top of the major axis to the camera and back to the bottomof the major axis. By angle here is meant the absolute value of theangle from the camera in any X-Y-Z direction where the Z-axis is outwardfrom the camera along the line of sight. From this we learn theorientation of the circular shape does not affect the measurement of themajor axis.

It is this fact that enables us to compute the length of the vector fromthe camera to the center of the detected oval. The tangent of one halfthe camera angle of the major axis is equal to one half of the detectedmajor axis divided by the length of the vector from the camera to thecenter of the oval. From this we deduce:

Major Axis=Distance from Camera to Center of lens|*(Tangent of cameraangle of Major axis)

Because the major axis is a fixed length:

D2=D1*Tan(angle1)/Tan(angle2)

Where D2 is the distance we want to find and D1 is a known measureddistance from the sensing device. Angle1 is the angle from the camerathe circle makes when at D1. Angle2 is the current distance from thecamera to the center of the ellipse.

Using these formulas, we are able to compute the distance from thecenter of the camera to the center of the lenses. If we know in advancethe camera's field of view we may compare this with known distances andcamera angles and use known measurements for a known camera field ofview. This eliminates the need for calibrating each camera. All that isneeded is the field of view of the camera. In certain cases, this may befound out from the computing device.

FIG. 26 shows how an embodiment of the instant invention calculates theX-Y-Z location of the lenses. The Z distance as measured from the screento the center of the lens is calculated by employing the distance fromthe camera to the center of the lens that was described in thedescription of FIG. 25. The angle from the sensor to the center of thelens described by the vector CC>CM is a known quantity. If the sensor'sfixed location is perpendicular to the viewing display and at the same Zlocation, then we have:

Distance from view screen to center of lens=(Distance from camera tocenter of lens)*Cosine(The angle the vector from camera to center oflens makes with Z-Axis).

Put another way: Z coordinate of eyewear lens=|CC>CM|*Cosine(Angle madeby Z-axis and CC>CM)

The last two equations are from the point of view of the camera andminor adjustments must be made if the camera eye is not in the sameplane as the viewing screen, or if the camera does not point directlyperpendicular to the viewing screen.

Furthermore, we now can calculate the X and Y coordinates by a similarprocess:

X coordinate of eyewear lens=|CC>CM|*Cosine(Angle made by Z-axis andCC>CM in the X direction)

Y coordinate of eyewear lens=|CC>CM|*Cosine(Angle made by Z-axis andCC>CM in the Y direction)

Now we must take these X and Y coordinates and translate them due to thefact the sensor is not located in the center of the screen, but this isan elementary task.

So, we have now described how to take the sensed images of ellipses andturn these into data for the X-Y-Z location. This is applied to thecircles of the lenses of the embodied eyewear to obtain viewpointinformation. This is then used to create perspective first and secondimages for one or more observers to create a pleasant viewingexperience. 3D stereoscopic images created from these viewpoints may beseen in stationary locations as described in U.S. Ser. No. 14/106,766and U.S. Ser. No. 14/547,555 as well as U.S. Pat. No. 8,717,423. Byutilizing proper discriminating techniques one or more viewers mayachieve 3D stereoscopic object interaction that is described in greaterdetail in the aforementioned and other patent applications.

FIG. 27 shows an embodiment of a secondary method for determining the Zdistance from the screen. Attached to our eyewear (item 2702) is nowdistance measuring equipment (items 2704 and 2706) or DME for short.There are numerous DME models available today. Some use sound waves andothers use IR or other light. Lightweight and inexpensive models arereadily available. One ultrasound detector currently available has arange from 10 to 450 centimeters with an accuracy of 0.3 of acentimeter. Sharp also makes an IR DME. The specifications and adescription of operation can be found on their web page. For both typesof DME a pulsed signal is sent from the DME that reflects off a surfaceand is measured on its return to the DME. One problem is determiningwhich surface the DME is reflecting off of. If the DME is located at theviewing display to measure the distance to the eyewear there will bemany objects for the signal to bounce off of and getting an accuratedistance to the eyewear becomes difficult resulting in inaccuratereadings which ultimately impact the images and hence the viewingpleasure in a negative way. In this embodiment, the DME is affixed tothe glasses. This gives it the broad target of the viewing screen toreflect the signal from. In this way accuracy of the distance from theeyewear to the viewing screen increases. This gives rise to moreaccurate Z location information and better quality images, hence viewingpleasure is increased. To further increase viewing pleasure, we mayexamine the angles of major, minor axis and make small corrections toeach lens Z distance.

In another embodiment, the DME may send information to the computingdevice via wireless means such as Bluetooth, however a wired method willalso work. A small battery may be employed to power the wirelessembodiment. A miniature transmitter along with a small computing deviceare additional devices which comprise this embodiment.

FIG. 28 shows an embodiment of a flow chart for both one and two camerasystems. In the one camera system, the circular shape is located andtracked. Then the center point, major axis and minor axis may bedetermined as desired. Next the length of the major axis is determined.From the length of the major axis the distance from the camera isdetermined. Using this information and the cameras view of the center ofthe circle or oval the X-Y-Z location in space can be computed. Ifdesired an approximate location of the eyeball may be determined formthe center point, major and minor axis data. This process is repeatedfor each lens of each user in order to compute the viewing perspectiveof each lens or eye for one or more observers.

The center of the lens or the actual eyeballs is then used to createreal world stereoscopic images. These real world stereoscopic imagesremain in an approximately fixed location in space as the viewpointvaries. As their location is stabilized in space they may now be mappedand compared with the location of real world physical objects to produceinteraction. This interaction may include, but not be limited totouching, moving, rotating, drawing etc.

In one embodiment of the instant invention a circular shape is used.From the X and Y data of the captured image an edge detector finds thecircular (elliptical) shape and determines the major axis and center ofthe circles. Because the circles are of known diameter and when tiltedthe major axis represents that diameter a comparison may be made whichenables the Z distance of the circles center from the camera to bedetermined. The vector from the center of the circle to the camera isalso used in the computation as an off-center circle will be furtherthan a centered circle in the same plane parallel to the camera andperpendicular to the cameras line of sight. So in this way all three,the X, the Y and also the Z distance may be computed from a captured 2Dimage.

An image captured by the camera may be processed with an edge detectorsuch as a “Canny edge detector.” For this reason, it may desirable inone or more embodiments to have contrast between the circles on theglasses frame and the background part of the frame. Now as an angle ofthe glasses in respect to the camera will cause the circles to look likeellipses, an ellipse detector is used. Several are known to be in use atthe time of this writing from “Hough transform”, “Akinlar and Topal toname a few. These are described very well on the web. In any capturedimage there will be several objects detected as ovals in addition to thedesired glasses lenses. Therefore, it is necessary to employ a softwarefilter using properties of the ovals that are representative of theglasses lenses. One such filter involves recognizing the two lensesovals will be similar in tilt and size. In addition, the distancebetween the ovals may be compared with the size of the ovals forparticular orientations. In this way, the data for the correct ovalsrepresenting the lenses of the glasses may be filtered out. From thefiltered data X and Y image information the actual X-Y-Z location of thelenses may be computed. In one or more embodiments, circular imagesaround the lenses are used, however other images X and Y captured imagedata may be used to compute real world X-Y-Z information with varyingdegrees of success. These images may be other geometric shapes but theymay also be a series of dots or blobs in a geometric pattern. Forexample, blobs of contrasting color may represent the four corners of asquare.

Another factor is the speed at which the glasses are moving when theimage is captured. Too high a rate of speed may result in a blurredimage that is not detected. To counteract this blurring, cameras withfaster shutter speeds may be employed. Another technique is to usesoftware to deblur the image. In addition, data smoothing algorithmssuch as a “Kalman filter” may be employed to mitigate the effects of arandom image blurred due to motion.

The instant invention allows all three coordinates; the X, Y and Z maybe derived from a single two-dimensional camera image. This eliminatesthe need for complex two camera triangulation techniques as employed by“Zspace” and also eliminates the need for IR distance measuringequipment such as employed with the Microsoft “Kinect.” Therefore, theinstant invention may be employed on systems with a single basic cameraas is common on most computer systems. This allows many more users toexperience the pleasure of real world stereoscopy on their personalcomputers. Cameras with faster shutter speed or IR light for lowlighting environments may be employed and a camera switching function isan option. However, the techniques described in the instant inventionare not limited to personal computers and may be used on larger and morecomplex systems such as those described in U.S. Ser. No. 14/106,766 andU.S. Ser. No. 14/547,555.

FIG. 29 shows an embodiment of a template for paper glasses (item 2900)employing a circular shape for tracking. Item 2902 is a white circularshape surrounded by a black frame. This embodiment is advantageousbecause there is only one circular edge that exists between item 2902and the frame (item 2900). This eliminates the need to discriminate fromother circular objects that might be on other designs of the frame.However, additional small circular or blob shaped objects may be placedat other locations on the frame to enable orientations with greater than90 degrees of tilt to be recognized and accounted for. Items 2906 arethe stereoscopic lenses which are used to discriminate among imagesreaching the eye. Examples include but are not limited to anaglyph(color discrimination), polarization (linear, circular, elliptical forexample), and shutter lenses. Items 2904 are blobs placed on the cornersof the stereoscopic lens. Due to the nature of the surfaces of thelenses there will be some reflectivity and hence a variation in colorsacross the lenses captured image. To reduce the possibility of falseellipses detected along the edges of the lenses these blobs createbarriers for the edge detectors. Additional blobs may be placed asdesired to reduce incidences of false ellipses being detected. Thisreduces or eliminates the problem due to the unknown contrast of variousreflections from the lenses. In addition, an anti-glare coating may beapplied to the lenses to reduce the reflections. In this way tracking isenhanced resulting in better tracking, better data for creating realworld 3D images and enhanced viewing pleasure. Item 2908 shows idealplacement for a company logo. The color scheme may be reversed with thedarker coloring for item 2902 against a lighter background frame. Inaddition, other colors may be used provided there is a contrast that maybe discerned by an edge detector when viewing the captured image. Inanother embodiment, the Canny Edge Detector is employed, however othermethods of detecting edges from the captured images may be employed aswell.

FIG. 30 is an illustration of an embodiment of a calibration tool. Oneof the greatest challenges for a tracking system is calibration. Unlessthe device employs fixed built in cameras of known location and cameraangle then calibrations must be made which take these parameters intoaccount. One method for making these calibrations involves tedious touchlocations that require the user to place their fingers in the locationof generated 3D objects and press a button when this is accomplished.This often requires multiple steps and often results in inaccuratecalibration. Reducing the stress of calibration while increasingaccuracy is possible by employing the calibration tool embodied in FIG.30. The tool consists of an arm (item 3002) and a diagram (item 3006).In another embodiment, two circles of known diameter (items 3008) areplaced in a known location on the diagram facing the camera. Themounting arm is placed over the camera lens at a 90-degree angle to thescreen. The camera is then made to capture the image presented on thediagram. The mounting arm is a known distance from the camera and theimage captured by the camera is analyzed. The filter will identify thetwo circles on the diagram and then the length of the major axis and thelocation of the centers can be determined. Based on this information thecamera width angle and the camera downward tilt angle can be determined.This information is then used to calibrate the software that determinesthe location of the glass lenses in X-Y-Z coordinates. Because thedistances are fixed and there is no guesswork in the location of thecalibration the viewer can quickly and accurately calibrate the glassestracking for their particular system, thus enhancing the viewingpleasure of the user.

FIG. 31 is an illustration of a method for 3D stereoscopic sculptingthat employs the glasses tracking system of the instant invention. Item3100 is the drawing canvas that may start out with none or one or more3D drawing pixels. These 3D drawing pixels may be referred to as“sculptels” or “voxels” in this document. As opposed to traditional 2Ddrawing pixels these sculptels have volume when viewed throughstereoscopic glasses. Furthermore, when the glasses tracking methodsdescribed in the instant invention are applied the sculptels take onreal world locations. These sculptels may be created or erased by movingthe hand or an object into their physical location in space. To bettersee the sculptels in 3D space, a texture may be added to the surfaces ofthe sculptel. For example, a dot, dots or a checkerboard-patternedtexture may be added. Random slight variation of the color of thetexture is also an option. The object's location is tracked using atracking device such as Leapmotions “Leap” device. It is then comparedwith the real world stereoscopic location of the sculptel. When theobject and the sculptel are within a predetermined range interactiontakes place. What happens during this interaction may be selected viauser control. Item 3102 gives some examples of the types of interactionsthat may be created. These include but are not limited to: drawing,erasing, coloring, changing transparency, changing the size of thedrawing, erasing or other tool. A “color wheel” consisting of aselection of blended colors may be employed. Additionally, the size ofthe sculptels may be selected to optimize it for the power of thesystem. The smaller the sculptels the more processing power is need. Oneway to alleviate the demands on the processor is to only draw thesculptels that will be seen by the user. A way to do this is to see ifthe sculptel has neighbors drawn. If all six sides are drawn then thesculptel will not be seen by the user and need not be drawn. The desireto do this varies of course when transparency of the sculptels otherthan opaque is employed. Other features that may be employed include theability to save, print, and edit created sculptures. Printing may forexample print in both 2D and 3D formats and convert between anaglyph andside-by-side (SBS) stereo formats. The side by side is the most commonstereo format for 3D television and projectors. Additionally, thesculpture canvas may be rotated about any axis and may also be made toauto-rotate. This allows pottery wheel like sculptures to be made asillustrated in FIG. 32. One or more tracked fingers may be used fordrawing or sculpting, either while still or rotating. In addition,drawing may be accomplished using the pointer beam created from thetracked location of the glasses and the glasses pitch, roll, and yawinformation. Depth may be changed using roll inputs about the Z-axis orby keyboard; mouse gamepad or other external inputs. Exclusivelykeyboard, mouse, gamepad, or other external inputs may also accomplishdrawing.

FIG. 33 is an illustration of an embodiment of the instant inventionthat allows two users to view displayed 3D real world objects from theirown individual perspective. Tracking two pairs of glasses and creatingimages for each of the four lenses that correspond to how the 3D objectwould be viewed from the perspective viewpoints accomplishes this. Theglasses then employ a combination two or more methods of discriminationto ensure the correct image reaches the correct eye of the viewer. Forexample, each lens would be both circularly polarized and colordiscriminating. This method is described in great depth in U.S.62/312,937 “Stereoscopic and Multiple View Glasses,” filed Feb. 24,2016. One of the primary advantages of this embodiment that allows twoviewers to see the same 3D real world object according to their uniqueperspective is as a teaching device. One of the individuals may point toan object and the other will see the same location being pointed to.This is not possible with the prior art in this field where a secondobserver would see a skewed image bearing little resemblance in form orlocation to what is seen by the first. This limitation is noted in U.S.Pat. No. 8,717,423 “Modifying Perspective of Stereoscopic Images Basedon Changes in User Viewpoint.” By removing this limitation, the abilityto teach is greatly increased, thereby adding to viewer pleasure. Inaddition, gaming devices where both players see the object in the samelocation increases the pleasure of the players.

In one embodiment 3D data from medical imaging devices can be importedinto the instant invention and a 3D real world image created using thesculptels of FIG. 31. This image could use different colors ortransparencies based upon the densities portrayed in the data from themedical imaging device. Because things such as bones have differentdensity than say blood vessels the sculptels can be grouped according todensities. In this way structures can be represented by sculptels withproperties that differentiate them from other structures. From this thesculptels may be grouped according to density that represent thestructures of the body. Once grouped they may be removed based on thegrouping property enabling structures to be made visible, invisible,transparent or of a certain color. In this way, the structures may beremoved by making the sculptels invisible and restored by making thesculptels visible. This allows the viewer to manipulate which structuresare seen allowing for greater flexibility. In addition, the structuremay be manipulated and made to rotate about any axis enabling a betterview of the desired structure to be seen. In addition to use as ateaching aid the instant invention may provide a diagnostic ability aswell as a sculpture for a physician to show his patients so they betterunderstand a diagnosis or medical procedure to be employed. Thus, theinstant invention can be used to enhance understanding and reduceanxiety of the patient.

This method may be applied to other fields outside of medicine as well.

FIG. 34 is an illustration of an embodiment of the instant inventionwhere the user interacts with the 3D real world object at a distance.This has several advantages as it eliminates the need to remove objectsthat are in front of the users fingers or pointing object. To enhancethe feeling of connection with the object being interacted with at adistance an imaginary electrostatic image (item 3406) is created betweenthe fingertips (item 3402) and the object (item 3404) being interactedwith. In this way the user can instantly tell a connection is madebetween his hand or fingertips and the object. The object is theninteracted with as though the hand or fingertips were in proximity. Toenhance the illusion the imaginary static electricity may be made tomove or vibrate and also cause a cue sound to be created by thecomputing device, thus simulating the motion of static electricity. Thefinger at a distance interaction can be enhanced with finger gesturessuch as touching two fingers together or moving them apart. This cancorrelate to grabbing and releasing an object. Other finger or handgestures may include but not be limited to twisting, turning,stretching, enlarging or shrinking objects. They may also be used tobring up a context menu. The finger at a distance interaction may beenhanced by keyboard, mouse or other inputs in conjunction withpointing.

FIG. 35 is an illustration of an embodiment of a laptop or foldingcomputer having a camera (item 3506) for eyewear tracking located in theupper portion (item 3502) and a camera or sensor (item 3508) for handtracking in the lower half (item 3504). The problem inherent in such aconfiguration is the angle between the face tracking camera (item 3506)and the hand tracker (item 3508) changes with the angle (item 3510) ofopening of the laptop-computing device. If the face tracking camera isused to create real world POV images that are to be coordinated withhand tracking to enable hand or pointer manipulations of the real worldstereoscopic images then the angular relationship of the two sensingdevices must be taken into consideration. So, a means to measure theangle between the top half and lower half of the open laptop must becalculated. In another embodiment, this may be done mechanically orelectronically with a sensor placed along the hinge (abeam item 3510) ofthe laptop wherein the measured angle is then sent to the software ofthe laptop so it always knows the opening angle. In another embodimenteither the image taken from a camera (item 3506) or hand-tracking sensor(item 3508) may be analyzed to give an approximate angle of opening. Atthe time of this application a company called Leapmotion makes a laptopwith a camera (item 3506) in the top half (item 3502) and also the handor object tracker (item 3508) built into the keyboard half (item 3504).Currently to use POV in combination with the hand tracker to manipulateobjects in real world space would require precisely positioning theangle (item 3510) between the top half and lower half of the laptopdevice. Another technique that may be used in one or more embodiments isto have a laptop hinge angle that locks in known angular positionsrelating the top and bottom halves. This information may then beemployed to relate the angle between the sensor in the lower portion tothe sensor in the upper portion. The instant invention eliminates theneed for precise angular adjustment thus increasing the user's abilityto enjoy 3D stereoscopic POV manipulations.

FIG. 36 is an illustration of an embodiment where the circles used fortracking are attached to a hat or other headgear. The size of thecircles that are tracked is a limiting factor in how far from the camerathey may be tracked accurately. There is a limit to the size of circleswhich may be employed with eyewear. For greater viewing distancesemploying POV computations it is beneficial to have larger circles totrack. A viewer (item 3602) who wishes to be at a greater distance fromthe viewing screen may have larger circles (items 3606 and 3608)attached to headgear (item 3604). The headgear may be a hat, band, orany object which is affixed to the head and moves as the head moves. Inthis embodiment, the location of the circles is determined in much thesame manner as for the circles of the eyeglass embodiment describedpreviously. To do this an adjustment is made for the larger size of thecircles. Then another adjustment is made which approximates the locationof the eyes in relation to the circles. It is true this will varyslightly from user to user because not everyone's head is shaped exactlythe same. However, an average location of the eyes in relation to thecircles may be used to give a fairly close approximation of where theeyes are for most users.

FIG. 37 is an illustration of another embodiment of the instantinvention. In this embodiment, the circle is used in a general distancemeasuring method or device. Most handheld phones (item 3702) todayemploy both a camera and a computing device or processor. By measuringthe size of the oval presented to the camera of the handheld phone andcomparing this with a reference size at a known distance the distance tothe center of the circle may be computed in the same manner as with thecircular glasses described earlier in this application. The measuringdevice is not limited to a handheld phone. Any device that employs acamera and computing device may be used for the measurements. Inaddition, most handheld phone devices also include tilt measurementdevices. In an exemplary embodiment, the tilt of the camera is used tocorrect the distance values and may also be used to also calculate the Yor height distance in relation to the handheld device. So, both thedistance and height information of the circular illustration in relationto the handheld device may be determined. In an exemplary embodimentthis information may be displayed on the display surface of the cameradevice in real time. Additionally, a zoom feature which is employed withmost camera devices may be employed for longer distances. Theinformation regarding the amount of zoom is used to correct the distancefor the fact that the zoom feature is employed.

In one embodiment, the circular shape (illustrated in FIG. 38) may beprinted from a computer image file. In another embodiment, the circlesmay be part of a pad with sticky material holding the pages of the padtogether.

In any given camera field of view there may be several items which arerecognized as ellipses by the ellipse detection software. It isimportant for the computing device to determine which ellipse to use forthe distance calculations. In this embodiment, a square object is placedoutside the circle and computer vision techniques are used to find bothcircle and square. A filter is then employed so that the circle insidethe square is used for measurement. While this is an exemplaryembodiment, the square may be placed inside the circle, other geometricshapes may be used in combination with the circle. In another embodimentobjects located inside or outside the circle may be used to discriminatethe chosen circle. In one embodiment, a smiley face or series of lines,curves, or blobs may be employed.

In FIG. 39 an embodiment is shown with two circles for distancedetection. In this embodiment the eccentricity, major and minor axesdetected by the handheld camera device will have nearly the same valuesand may be used to filter the chosen ellipses from other backgroundellipses. In an exemplary embodiment, the center of two circles is thedistance measured to, but other location such as the center of one ofthe circles may be used.

FIG. 40 shows a flow diagram illustrating an exemplary embodiment of theprocess for determining distances to a flat circular object. In otherembodiments, other geometric figures or series of figures may beemployed. However, the circular figure is advantageous due to thespecial properties of a circle that make the calculations easier.

First, the ellipses are detected within the field of view of the camera.The camera captures the image. A filter may be used to select thecorrect ellipse or ellipses to be used for measurement. These may employellipse pair filters or other geometric features such as the circlewithin a square technique.

The major and minor axes are determined in pixels of the camera image.The center of the ellipse(s) is determined in pixels of the cameraimage. This data is corrected for camera tilt angle and anymagnification of the camera. The order of the calculation steps is notimportant and any order that achieves the end result may be employed.

Alternatively, for auto stereoscopic systems the head and eyes or otherfacial may be tracked. The distance between the eyes is then used tocalculate depth distance in much the same way as the major axis is usedwhen tracking ellipses. However, this presents a problem. When the headis rotated about the yaw (Y-axis) the distance between the eyes in theX-direction needs to be compensated for the yaw. Else the distancebetween the eyes (or other facial features) is now less than thedistance as seen from front on view of the face. The head yaw angle maybe determined using computer vision and a trigonometric adjustment tothe eye distance may be made. This ensures the correct Z distance to theface and eyes may be computed.

The pixel coordinates are used to determine the size and location inrelation to the camera-viewing field.

As the viewer moves further from the camera it becomes more difficult totrack objects and extract data. For this reason, an optional camera zoommay be employed. The zoom may be controlled by parameters entered intosoftware and the processor makes adjustments. The area zoomed in on neednot be centrally located.

This data is then compared with known size and distance locations. Forunfamiliar camera types the calibration tool may be used to calibratethe camera for computations.

First absolute distances to the detected ellipses are determined. Thenthe angular values are used to compute X, Y, and Z values as desired.The Pythagorean Theorem is useful when applied in three dimensions toobtain these results.

In this way, the real-world distance values may be obtained.

The results are then applied to the specific use. They may be used as 3Dimaging camera point of view (POV) coordinates. In this way POV 3Dimages may be created. By creating 3D POV images for each eye andemploying stereoscopic techniques 3D POV stereoscopic images may becreated. The stereoscopic techniques which may be employed include, butare not limited to shutter glasses, passively polarized glasses,anaglyph glasses, and auto stereoscopic systems. These are well knownand developed at the time of this application.

Another use involves distance measurement equipment. The distance maythen be displayed as distance to the target circle or circles and alsothe height may be computed in this embodiment. This has uses in theconstruction field, home improvement, golfing and many other fields. Thetruly remarkable feature of this distance measurement method is it doesnot require any tools other than a handheld camera attached to acomputing device. Most people these days carry such a device in the formof a cellular phone so they will usually have it when it is needed. Thecircular object can be printed from a file on their phone, computer, orfrom the web. Therefore, this embodiment is convenient and easy to use.

With reference now to FIG. 41 an exemplary embodiment shows how theglasses may be employed in a multi-user viewing environment. Todifferentiate between users the geometric object (in this embodiment acircle) and its background alternate between light and dark. In thisillustration four different combinations are shown, however there areother patterns and combinations of patterns that may be employed. So,for example using computer vision one pair would be recognized as havinglight objects on dark backgrounds. A second pair may have dark objectson light backgrounds. Third and fourth pairs if desired may havealternating light and dark foregrounds and backgrounds. The software andprocessor may employ filtering techniques to determine which images areto be prepared for viewing by each coordinated lens. The lenses havemultiple discriminating features as described elsewhere in thisapplication to ensure only the coordinated image for the lens passesthrough to the viewers' eye(s).

With reference now to FIG. 42 is an illustration of an exemplaryembodiment of glasses with added tracking dots or blobs (items 4210,4212, 4214, and 4216) in the four corners of the front facing surface.These dots may also be referred to as blobs within this document. Thedots are located in the four corners and may be located either in frontof or behind the plane that has the circular tracking objects (items4206 and 4208). They may be employed together with the circular objectsto determine the angular components of the glasses. In particular, thetilt, roll, and yaw may be computed. This shall be further explainedlater in this document.

In another embodiment illustrated in FIG. 42 the dots are part of a foldmade from the front surface (item 4222) and the template surfaces (items4226 and 4224). In this embodiment, a cut is made that allows thesurface the dots are on to be pushed backwards towards the user. Thisleaves a gap or hole (item 4218 and 4220) where the cuts were made. Partof the surface of the front plate (item 4230) is pushed backwards atapproximately a 90 degree angle in such a manner so that part of thetemplate containing the dot (item 4210) is now parallel and behind thefront plate. This may be repeated for all four corners. However, onlyone tracked dot is needed in addition to the circular tracked objects toperform the roll, yaw, and tilt calculations.

In this embodiment items 4206 and 4204 contain light filtering materialused to discriminate between left and right images so each eye receivesthe coordinated stereoscopic image created for its point of view (POV).

With reference now to FIG. 43 an illustration of the rotationalmovements the head can make is shown. Being affixed to the head theglasses rotate as well. In an axis system for the head the up and downdirection from the body to the head would be the Y-axis, from ear to earthe X-axis and from in front of, to behind the head the Z-axis.

The head may move to the right and left in similar movement to oneindicating a “no” response. In this case, the head rotates along theY-axis and this may also be referred to as “yaw.” Head movement tiltingup and down similar to someone nodding yes is rotation about the X-axisand may also be referred to as “tilt.” Head movement from shoulder toshoulder is rotation about the Z-axis and may also be referred to as“roll.”

FIGS. 44A, 44B, and 44C illustrate a method for using the tracked dotstogether with tracked circular objects to calculate pitch, roll, andyaw. These figures illustrate glasses in different orientations as seenfrom the point of view of the tracking camera. FIG. 44A is anillustration of glasses facing directly at the camera wherein the frontplate of the glasses is approximately perpendicular to a line drawn fromthe camera to the center of the front plate. FIG. 44B is an illustrationof glasses affixed to a head that is tilted downward about the X-axis.FIG. 44C is an illustration of glasses affixed to the head that isrotated or yawed about the Y-axis to the user's left (or the trackingcameras right).

Before we begin the explanation of calculations for yaw, roll, and tiltit should be noted that the methods described elsewhere in this documentexplain how the circular objects are employed to calculate the distancefrom the camera of the glasses. This is an important step in thecalculation of the angular positioning of the glasses of thisembodiment. It is possible and within the scope of the invention to usedots, blobs or other shapes in place the circles that combined with thedots located on another plane enable the same calculations of distance,location, tilt, roll, and yaw. These require more complicatedtrigonometric calculations, but a skilled mathematician can make themand therefore they are within the scope of this invention. The same istrue in a glasses free system. Facial recognition technology enablestracking of eye location as well as pitch, roll, and, yaw of the head.By combining standard or entered values for the distance between theeyes along with yaw about the Y-axis the distance to the user's head oreyes can be determined. Thus, all of the required information for thecomputations of a glasses free system are available and similar methodsfor glasses free 3D POV systems with or without a pointer or controlleras described herein are within the scope of this invention.

With reference to FIG. 44A we see an illustration of an embodiment forhead tracking glasses that have the circular tracking markers (items4414 and 4416), additionally there are four added tracking dots or blobs(items 4402, 4404, 4405, and 4406) in the corners of the faceplate (item4410). The corner dots are located either in front of or behind theplanar surface (item 4410) where the circular tracking markers are. Oneor more imaginary line(s) or line segment(s) (items 4408, 4410) areconstructed between the dots or blobs. An additional imaginary line(item 4412) is created between the center axis of the circular trackedobjects. Any or all of these line segments may be mathematicallybisected and perpendicular lines may be constructed along the point ofbisection. These lines and line segments are then used for comparisonamong each other.

To accomplish this the captured image of the glasses is analyzed usingcomputer vision software such as OpenCV. The location of the circles (orcaptured ovals) and dots are then computed using software and aprocessor. Then trigonometry is used to create equations for the line(s)and/and or line segments. Distance from the camera may be computed usingthe circular method described in this application or trigonometry may bedirectly applied to dots or other shapes by comparing distances betweenobjects in the captured images with the known distances between objectson the glasses. These methods may also be applied to a tracked glassesfree head or to other objects with tracking markers affixed to the headand these systems fall within the scope of this application but shouldnot be considered limiting.

There are new glasses free auto stereoscopic systems which have beendeveloped. One such system has been created by 3DTau. Their systememploys filters that direct the light to discrete locations in the Xdirection. They are all presented at once through a filtering technique.They also emphasize that head/POV tracking is not needed nor employed.They take multiple camera angle images and present them all at oncethrough various viewing angles.

However, their system is limited in that views above or below are nottaken into account. In addition, it does not take the distance from theviewer into account.

Due to these limitations, it cannot be used as a real world stereoscopicsystem that allows a user to interact with the stereoscopic images.However, one or more embodiments of the invention presented here may beemployed to modify the system of 3DTau and others like it to overcomethese limitations.

Computer vision techniques enable eye tracking as well as finding pitch,roll and yaw orientation of the head. An example of this is found in“FacetrackNoIR.” The eyes and other facial features can be found easilyusing openCV. The distance between the eyes or other facial features canbe used to calculate the distance from the sensor. By adding thisinformation and image generation on the fly to the system of 3DTau orother auto stereoscopic systems, one or more embodiments of the systemmay create real world images that can be interacted with. The hands-freecontroller of FIG. 42 can be employed in a similar fashion by replacingglasses X-Y-Z location and orientation information with head X-Y-Zlocation and orientation to create a head oriented controller. Hence oneor more embodiments of the system may include the use of face trackingto enhance glasses free auto stereoscopic systems.

In the orientation shown in FIG. 44A the glasses are presented to thecamera with no tilt, roll, or yaw. It can be seen that the line throughthe centers of the circles (item 4412) bisects the vertical linesegments (items 4408 and 4410) and from this we can deduce there is notilt about the X-axis. We can also compare the perpendicular bisector ofthe circles with the perpendicular bisectors of the horizontal linesegments (not shown) connecting the dots. This will give us yawinformation about the Y-axis.

Moving on to FIG. 44B we see an illustration of an embodiment of theinstant invention where the head and/or glasses are tilted downward inrelation to the tracking camera. Now the bisector of the vertical linesegment between the dots (item 4432) is above the line (or line segment)connecting the circular objects. This separation between the linesegments may then be measured. Distance from the camera is known andtrigonometry may be applied to calculate amount of tilt about theX-axis.

With reference now to FIG. 44C, the glasses are shown yawed about theY-axis. Horizontal line segments between the dots may be computed andtheir perpendicular bisectors may then be compared with theperpendicular bisector of the line between the centers of the circularobjects. By applying trigonometry, the amount of yaw about the Z-axismay be computed.

The computations for roll about the Z-axis are simpler to compute asthey only involve comparison of the center points of the circularobjects.

It should be noted that there are many ways of applying trigonometry tosolve equations for pitch roll and yaw and the methods described in theexemplary embodiment are not intended to be limiting. In addition, thecomputations may be performed using as few as one dot out of the planeof the surface of the glasses. It is even possible to obtain theinformation from a single dot located on the plane of the glasses byanalyzing the major and minor axis of the camera-captured ellipses. Bycomparing previous location with current location, the information maybe obtained with just the circular tracked objects or two or more dots.So, there are many ways to extract the information. They all fall withinthe scope of this invention and the embodiments shown while exemplaryare not intended to be limiting.

With reference now to FIG. 45 an illustration of an embodiment of apointing system or method is shown. This pointing system employs thedistance, pitch, yaw, and roll information to enable a 3D pointercontrolled by the glasses, the head, or other tracked object affixed tothe head. An image capture device (item 4520) is located in fixedrelationship or with a known relationship to the display unit (item4516). In the exemplary embodiment portrayed a 3D stereoscopic line isdrawn on the display (item 4516) so the user(s) eyes (items 4502 and4504) see a line emanating from the center of the glasses (item 4510).In embodiments that employ glasses the lenses (items 4506, and 4508) aremade in such a way as to ensure the correct stereoscopic image reachesthe coordinated eye. Stereoscopic methods employed may include passivelypolarized glasses, shutter glasses, anaglyph glasses, combinations ofthese methods and auto-stereoscopy. In this embodiment, the line isdrawn from the center of and perpendicular to the front plane of saidglasses. The line is continually updated to reflect the current state ofpitch or yaw of the glasses and remain perpendicular to front plane. Byrotating the head about its axis, the user can control the direction ofthe line. As the location of the line in real world space is known itslocation can be compared with the 3D POV real world stereoscopic objects(item 4518). When the line intersects, or is near the 3D real worldobject interaction may occur. Additionally, other controlling devicessuch as keyboards, computer mice, trackballs, and gamepads may be usedto enhance the interaction capabilities of the line and object. Theobject may be captured and made to translate location, rotate, expand orcontract, change transparency or texture or be made to perform othertasks. The software writer only decides the limitations. They may alsobe used to control external devices. Examples might include hands freecontrolling flow and temperature of water in a restroom. Other uses mayinvolve places where germ transmission is a consideration, such as ahospital.

Other additional features include a pointer reference marker (item4522). This is drawn in front of the glasses along the line drawnperpendicular to the glasses. It may be employed for several functions.One function is as a calibration device. The location of the marker isdrawn in relation to the glasses. This may then be compared with thelocation of 3D stereoscopic objects with known location in relation tothe display. By movement of the head, the user can place the marker(item 4522) in the same real world location as the stereoscopic object(item 4518). The processor can use this information to calculate acorrection to be applied to correct for any difference in usersperceived location with the currently computed location.

Additionally, the marker may be moved in relation to the glasses usingkeyboard or other commands to adjust for any calibration errors. Themarker (item 4522) may be made to move towards or away from the user'sglasses by rolling the head along the Z-axis or external keyboard ormouse inputs.

The marker (item 4522) may be any shape and may be made to emulate anairplane, gun, slingshot, projectile firing device, flying animal, fishor other animal, car or other vehicle or just about anything that can beimagined. It has applications in medical imaging devices as well asgaming devices and drawing or sculpting device. In the case of a gamingdevice projectiles may be made to project outwardly along or near theline emanating from the glasses (item 4514) towards the display (item4516). These 3D stereoscopic projectiles may then interact with 3Dtargets created by said display. The descriptions of these embodimentsare not meant to be limiting in any way and the scope of the methods anddevices is what is in the claims.

Additional line segments (items 4512) may be drawn to help with userdistance judgment regarding the location of the marker.

To summarize, four methods were presented for determining the distancefrom a sensor to eyewear with geometric lenses. One embodiment employscircularly shaped lenses and a single camera or sensor. Otherembodiments include, but are not limited to two-sensor triangulation tothe eyewear, plenoptic cameras or sensors, and distance measuringequipment mounted to the eyewear.

In another embodiment, the eyewear employs circular features oncontrasting backgrounds. These features are exterior to the actuallenses in one or more embodiments. The features may include other shapessuch as a square and/or multiple objects. Blob patterns of contrastingcolors may also be employed.

Electroluminescent materials may be used to create the image contrastsfor object tracking. This may be especially useful for dark viewingconditions. In addition, a fast strobe pattern may be synchronized withthe speed of camera capture to reduce blurring of the captured images.

The use of circular objects for distance measuring may be employed byitself for applications used on tablets, cell phones or handheldcomputers. Additionally, telephoto lenses may be employed to increasethe range of distances that may be measured. These telephoto lenses arecurrently readily available to attach to most common cell phones or tothe cell phone case.

For autostereoscopic systems, facial recognition software may measurethe location of eyes and other facial features. Additionally, facialrecognition software is capable of calculating pitch, roll and yaw ofthe head. By employing trigonometry and an entered or assumed value forthe distance between the eyes the distance to the head may be computed.Yaw and the distance between the eyes in the captured image provide thenecessary information to make these computations. In this way, all ofthe features described for the head tracking eyewear and eyewearpointing device may be employed in an auto stereoscopic system.

In another embodiment, the perimeter of the lens employs methods toenhance contrast. It has been explained how to employ UV or otherilluminating source to enhance contrast in order to enhance tracking bya sensor.

Properties of circles and ovals have been taught. Once these conceptswere explained it became possible to teach the trigonometric algorithmsthat enable X-Y-Z data to be determined by means of tracking thecircular lenses with a single sensor. As explained in this description,the ability to use a 2D captured image to compute X-Y-Z coordinates maybe extended to other geometric objects or group of objects. The scope ofthe instant invention includes applying the principles described in thisdescription to any 2D captured image of objects of known separation,width or height that may be used to determine the X-Y-Z data for lens oreye tracking as is required for creating correct perspective images andwithout the need for dual camera triangulation methods.

The advantages of distance measuring equipment attached to the eyewearrather than the viewing display has been explained.

One big advantage of the instant invention is all three axis X, Y, and Zof the user's approximate eye location may be obtained from a singlecamera of the type currently employed in most computing systems.Extraction of 3D Z depth information from a 2D flat object is applied tothe method of 3D POV stereoscopy to produce superior results at a smallfraction of the cost of current systems. This will enable many morepeople to afford and derive pleasure from POV 3D stereoscopicapplications. In addition, this may also be applied to non-stereoscopic3D POV devices as well. This has application to the 3D gaming industry.It also has application to 3D drawing programs such as Vectorworks® andAutocad®. Additionally, it may be applied as a teaching device and formedical imaging. Two viewers would be able to see the same 3D objects(stereoscopically or not) from vantage points created for their POV.This prevents the image from appearing skewed to the second user as isthe case with other devices at employed at the time of this application.

Another advantage of the instant invention is pitch, roll and yawinformation of the users may also be obtained from a single camera ofthe type currently employed in most computing systems. This informationmay be employed to create a 3D stereoscopic “beam” emanating from theglasses that may be employed to interact with other 3D stereoscopicobjects. In addition, the processor may create 3D stereoscopic objectsto be displayed in front of the user's head. These objects may followthe user's head movements. For example, an airplane's pitch, roll andyaw may be made to follow the pitch roll and yaw of the user's headgear.Headgear may be glasses, hats or any other object affixed to the headthat moves as the head moves. Guns may be aimed naturally by tilting thehead in various directions. The target image in front of the glasses maybe made to vary forwards and backwards by rolling the head from side toside about the head's Z-axis. Other commands and interactions may becreated depending on the desires of the software writer and this list isnot intended to be limiting in any way.

While the invention herein disclosed has been described by means ofspecific embodiments and applications thereof, numerous modificationsand variations could be made thereto by those skilled in the art withoutdeparting from the scope of the invention set forth in the claims.

The headings used herein are for organizational purposes only and arenot meant to limit the scope of the description. As used throughout thisapplication, the word “may” is used in a permissive sense (i.e., meaninghaving the potential to), rather than the mandatory sense (i.e., meaningmust). The words “include,” “including,” and “includes” indicateopen-ended relationships and therefore mean including, but not limitedto. Similarly, the words “have,” “having,” and “has” also indicateopen-ended relationships, and thus mean having, but not limited to. Theterms “first,” “second,” “third,” and so forth as used herein are usedas labels for nouns that they precede, and do not imply any type ofordering (e.g., spatial, temporal, logical, etc.) unless such anordering is otherwise explicitly indicated. For example, a “third dieelectrically connected to the module substrate” does not precludesscenarios in which a “fourth die electrically connected to the modulesubstrate” is connected prior to the third die, unless otherwisespecified. Similarly, a “second” feature does not require that a “first”feature be implemented prior to the “second” feature, unless otherwisespecified. As used in this specification and the appended claims, thesingular forms “a”, “an”, and “the” include singular and pluralreferents unless the content clearly dictates otherwise. Thus, forexample, reference to “a linker” includes one or more linkers.

Various components may be described as performing a task or tasks, forconvenience in the description. Such descriptions may use the phrase“configured to.” Reciting a component that is configured to perform oneor more tasks is expressly intended not to invoke 35 U.S.C. § 112,paragraph six, interpretation for that component.

What is claimed is:
 1. A method for calculating position in relation toa camera comprising one or more ellipses; a camera configured to captureimages of said ellipses; an image analysis system coupled to said cameraand configured to calculate a position of each of said ellipses fromsaid camera images; and, said calculate a position and an orientation ofeach of said ellipse comprises locate a two-dimensional projection ofsaid ellipses in said camera images; and, measure the length of majoraxes of said ellipses in said camera images; and, calculate a distancefrom said camera based on size of said major axes, and calculate athree-dimensional position and orientation of said ellipses from saidtwo-dimensional projection.
 2. The method for calculating position ofclaim 1, wherein said ellipses are attached to an article that is in afixed location in relation to a person.
 3. The method for calculatingposition of claim 2, wherein said fixed location is in relation to thehead of said person.
 4. The method for calculating position of claim 1,wherein said camera is a built in or stand-alone camera that capturesstill images and video on a computer.
 5. A method for calculatingdistance in relation to a camera comprising one or more ellipses; acamera configured to capture images of said ellipses; an image analysissystem coupled to said camera and configured to calculate a position ofeach of said ellipses from said camera images; and, said calculate aposition and an orientation of each of said ellipse comprises locate atwo-dimensional projection of said ellipses in said camera images; and,measure the length of major axes of said ellipses in said camera images;and, calculate a distance from said camera based on size of said majoraxes.
 6. The method for calculating distance in relation to a camera ofclaim 5, wherein said ellipses are attached to an article that is in afixed location in relation to a person.
 7. The method for calculatingdistance in relation to a camera of claim 6, wherein said fixed locationis in relation to the head of said person.
 8. The method for calculatingposition of claim 5, wherein said camera is a built in or stand-alonecamera that captures still images and video on a computer.
 9. A positioncalculating system comprising one or more ellipses; a camera configuredto capture images of said ellipses; an image analysis system coupled tosaid camera and configured to calculate a position of each of saidellipses from said camera images; and, said calculate a position and anorientation of each of said ellipse comprises locate a two-dimensionalprojection of said ellipses in said camera images; and, measure thelength of major axes of said ellipses in said camera images; and,calculate a distance from said camera based on size of said major axes,and calculate a three-dimensional position and orientation of saidellipses from said two-dimensional projection.
 10. The positioncalculating system of claim 9, wherein said ellipses are attached to anarticle that is in a fixed location in relation to a person.
 11. Theposition calculating system of claim 10, wherein said fixed location isin relation to the head of said person.
 12. The position calculatingsystem of claim 9, wherein said camera is a built in or stand-alonecamera that captures still images and video on a computer.